Testing Software Series: Integration/Smoke Tests

Tests in general, are overhead. A waste. Extra code to create. Extra code to maintain. Extra code to extend. They are only valuable to a business if they can be cost justified.

So the question begins, what tests can be cost justified?

This entry is dedicated to Smoke Tests/Integration Tests.

Smoke tests/integration tests ensure that your product is doing what it’s supposed to do, more from a customer standpoint or high level API (like a RESTful interface). If you added a new library/dependency to your project, they’ll ensure your product still works. If you upgraded a dependency, same thing.

These tests ensure you REMAIN productive, whether you’re first starting to build your product/API or trying to extend an existing.

IMHO, these are the most valuable tests you should have, they should be the most extensive, and hopefully, they are the first tests you write. Nothing says Test Driven Development (TDD) better than writing some high level tests on what you want to accomplish before you start. Before you start wiring up databases, before you break up your architecture into “Micro Services” or whatever the new hype will be. These tests make sure your SYSTEM works so it’s USABLE by the customer.

Now, back to the cost justification. Yes, they are a waste, but without them, your risk is much higher for shipping software that doesn’t work. So first, they act as an insurance policy, limiting some of the heavy downside/asymmetric risk to failed software in the field.

Justification number two. After some initial time to set up, they will actually speed up development. This is possible because 1) they keep you from having to manually test use cases again as you continue to extend/refactor the underlying code, and 2) they provide UP-TO-DATE documentation on how to use the code at a high level, so it you are splitting up work/separation of concerns, or have been away from the code for some time, the party adding onto the API can look at the tests, and know how to call the API/wire it up.

Justification number three. They remove the need for manually testing, and re-testing by a designated tester, or at least minimizes this workload. Trading a day or two of creating scaffolding/setup that can be re-used for other API’s/test suites, for a day or two of manually testing, is practically an immediate payback!

Overall, from a cost perspective, point one on insurance policy is probably somewhere between a few months to a year or two payback tops. Somewhere in that time frame, these tests are going to save you from the much added expense of handling large failure(s) that happen to customers, that then take tech support, engineering support (unscheduled, on-demand support), and customer frustration until the issues are fixed. Point two on speeding up development, the breakeven from this point alone is most likely a few days. A return of thousands of percent a year. Where can you normally make that from an investment? And the last point on removing need for manual testing. Immediate payback. Infinite return.

End of the day, I see no reason to NOT have these tests. These integration/smoke tests should be the first thing your setting up BEFORE YOU START creating a new API, as to maximize your return from the onset.

Leave a comment