QA organisations have come a long way in the last decade or two. In the early 2000s, automated unit testing was just gaining popularity and QA was performed manually by armies of testers. The QA process was time-consuming and arduous, and the testing timelines were the first thing to be compressed when development took longer than planned (as it almost always did).
By the 2010s, many applications had moved to a web interface which allowed QA teams to automate some or all of the end-to-end testing using Selenium, or similar. This was a giant leap forward, especially for basic 3-tier apps with a web UI, a “services layer”, and a database. Everything was nicely contained in the one codebase, including the tests. Coupled with Cruise Control Hudson Jenkins, we had a reliable way to delivery quality software quickly.
Then came microservices, APIs, and connected applications. This architecture requires another shift in QA strategy that I see many large enterprises struggling to adopt. Applications built on a distributed model cannot be reliably or adequately tested in a connected environment - there’s a combinatorial explosion of downstream dependencies outside of the control of the QA or Dev team, and any one of these dependencies could be offline at any time and bring your QA process grinding to a halt.
The complexity of setting up test data setup in a distributed system is even more challenging. Creating the necessary data to test a single scenario often requires multiple support tickets, data conditioning specialists, and exclusive locks on test environments. It sucks up time and resources, and causes the testing process to drag on.
Despite this, many large enterprises continue to cling to the idea that integrated testing is the only way to ensure quality. It’s not. There are better alternatives.
By combining API contracts, mocking, and automated API testing, each part of a distributed system can be tested independently. Your web app should be tested against mocked services that comply to your API contracts. Your API should be tested using API tests to verify that they comply to their published contracts. API changes should always be backwards-compatible.
A QA strategy such as this takes courage and confidence to implement. It needs to have buy-in from the product team, and executives, and most importantly the development teams need to comply to it. APIs without contracts are no longer acceptable. Devs need to build backwards-compatibility into their APIs. QA engineers need to understand how to use tools like Wiremock or one of the myriad of options in NodeJS, and how to write reliable automated tests in any number of frameworks or languages.
If your QA team are still relying on testing in an integrated environment then it’s almost certainly the biggest bottleneck to frequently and reliably shipping updates to your customers. It’s no longer acceptable to have a multi-month QA process - you need to reduce that to a few hours to be competitive, and it all starts with a clearly-articulated QA strategy spanning the entire organisation.