I was at a software QA conference a few weeks ago and attended a presentation on implementing a test automation framework. The presentation was skewed toward hard ROI as the only measure of success. While I acknowledge the importance of the hard ROI focus on cost savings in tools, time and people, it unfortunately didn’t speak to some of the intangibles that solid automation programs can create within teams.

I’d actually like you to broaden this view when considering implementing automation, from beyond these hard savings to adding a few softer intangibles that can have a huge impact on your automation value proposition.

There are really three keys to focus on:

• Speed of testing cycles and overall project cycle time
• Increase of coverage and customer-visible quality
• Freeing time within testing teams to invest in continuous improvement

Automation can drastically improve your cycle time speeds if done well. From a design perspective, you’ll want to make the test run small and independently for ease in execution and to reduce re-execution overhead. You’ll also want to pay particular attention to ease of failure diagnosis. In a large automation effort I was a part of, one problem we discovered was the amount of time test failure root cause analysis was taking. It was actually taking us longer to analyze the results than to run the automation.

This cycle time improvement can help differentiate your team in another way. I’ve used it within agile teams to support iterative, frequent development releases—to create a safety net, if you will, that could run fast regression tests for late-breaking application evolution based on customer feedback. This support allowed the development team to be highly adaptable in responding to feedback without increased quality risks. And our “customers” loved the speed.

As you increase your coverage via automation, you want to make it visible to all stakeholders. Talk in terms of increasing test frequency and increased coverage. That coverage can be viewed in a variety of value-added ways:

• Uncovering defects faster – to increase throughput of repairs
• Uncovering variation in application component maturation rates
• Increasing your overall application coverage levels and ability to rerun tests to overcome regressions and blocking issues
• Active reporting of coverage levels (tests planned, actually run and results) as a risk awareness tool

Simply put—increasing your coverage depth, breadth and connection to customer expectations can be a huge deal in delivering on your project commitments. It also helps to provide more real time guidance levels to the project so that adjustments and course corrections can be made.

Finally, automation will free up time within your team. You have a choice to factor this time into cost savings or realizing hard ROI factors, or you can reinvest this into making your team more effective. I much prefer the latter approach. But where are good opportunities for reinvestment? I’m glad you asked. The first place is into more automation. Good automation takes time to develop. You need to create an infrastructure and conventions to ensure it grows well. You also need to consider maintenance effort—driven by tool updates and ongoing application evolution.

Another area is simply investing in your team. Increase the technical training of your team to increase your competitiveness. Don’t forget leadership and soft skills. Finally, reinvest in the processes and approaches you use for overall testing. For example, add exploratory testing techniques to your skills base for a time when speed is required but automation isn’t appropriate or ready. Or improve your coverage planning so that it better aligns with the development team’s product evolution goals.

These three core soft ROI factors can truly differentiate your testing team and your overall contributions to product quality. They should be added to your hard factors to create a balanced evaluation and justification of any automation effort. Beyond that, though, the hard ROI models have too simplistic a view that can sometimes lead to arbitrary cost cutting and staff reduction in SQA teams.

While I’m all for saving money, it should be balanced in approach and not made based on a single analysis point. Quality isn’t always simply about the numbers. We need to get that point across!

Bob Galen is an author, speaker and the principal consultant of Cary, N.C.-based RGCG LLC (www.rgalen.com).