ro·bust·ness: the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions.
I had the bittersweet experience of watching my app break in a demo, bitter because everyone was watching and sweet because it was a demo. The failure resulted from my assumption that the wireless network would be dependable, which sounds silly now in retrospect. Proper error handling would have allowed the app to fail silently instead of exiting with an exception which is what it did. Not only that but it would have been great for me to have the app pick up where it left off when the network was restored; instead I was obliged to perform manually the task left unfinished by the premature exit.
What I should have done is break my design pre-emptively. Tests for robustness are a priority, not a nice-to-have. I offer a few tips based on my experience …
1. The Test Environment != Reality
This one is a doozy – don’t assume that since the app works in the test environment that it will work under the stress of real circumstances. Try to test in the field if at all possible.
2. Write tests with assertions you don’t expect
Testing for expected inputs isn’t what will tell you whether the app will fail. Test for the unexpected inputs which could actually break the app. The weirder the better.
3. Document Everything
The more you write about your code, the more chance you will see the failed logic you missed the twenty previous times you looked at it. Teach the technique in a blog post or a tutorial, and make sure to write readable code with comments.
How do you test for the “real world”? Or should we let the users drive the fixes?