Complex Problems don’t always need Complex Solutions
About 10 years ago I was running a Testing Consultancy and I had won an assignment at a major utilities company where they wanted to reduce the costs of testing their new System Releases by 20%, but without significantly impacting the overall quality.
They believed that this was going to be complicated and time consuming, so the initial piece of work was a 20 day assignment to come up with an approach to solving the problem, followed by the implementation of my recommendations.
When I started, I asked for all the information that they had on the testing they did: the number of testing phases; the costs per phase; number of test cases executed per phase; and number of defects found per phase.
They told me that had the data I requested, but not in a consolidated report that would be easy read, but they definitely had the raw data so I could create the report for myself.
I decided to spend the first morning creating the report to get a better understanding of their situation.
They had several phases of testing: System Test, Integration Test, User Acceptance Test, Regression Test I and Regression Test II.
For the purpose of this post, the difference between the different types of testing doesn’t really matter, as we’re looking at the costs and effectiveness of each phase.
After about 2 hours of analysing the data I created the following chart, which I shared with the management team, to ensure I had good understanding.
When I presented the Report Summary, the management team said that the Information I had presented was accurate and inline with their understanding of the costs and effectiveness of each phase.
They did tell me that they were surprised that I had called the meeting on the first day, just to go through the chart.
They felt it could have waited a week or two until I had some idea of how I was going to resolve their complex problem.
Reducing testing effort by 20% was no easy task, especially if you wanted to ensure that there were was no real drop in quality.
The problem was though they were looking at things from the wrong angle, they assumed that their Complex Problem needed a Complex Solution.
Whereas I was looking to see if there was a simple solution first.
I told them that I think I’ve found a Simple Solution, and one that was very easy to implement, fully saves the 20%, small impact on overall Quality, and probably at zero cost to implement.
They were stunned when I told them, looks of disbelief all over their faces.
So I showed a second chart I had prepared.
The second chart was the same as the first chart but with just the Regression Test I highlighted.
I said why don’t you stop doing Regression Test I, it costs 20% of your budget, it’s only finding 2% of your defects, so it’s not very effective.
Also, your Regression Test II includes 5 million customer accounts, so there is a good chance that the defects missed in Regression Test I would be found here.
If we take this approach we can reduce your costs be 20% and still have probably 99% of the defects found, so not a significant drop in quality.
They discussed the idea for about 20 minutes and then finally agreed with my proposal.
So now we had a Simple Solution, for what had been perceived as a Complex Problem.
Unfortunately, for me they also ended the assignment too, which means I just go paid for one days work, rather than the full 20 days.
Seem they had found another simple way to reduce cost without impacting quality 🙂
Gordon Tredgold
GET THE COMPLIMENTARY E-BOOK
