Automation Payback
By Ray Claridge
I’ve previously blogged about automation and the need to ensure real payback value, so let me fill you in on a simple elaboration technique to prove if automating is worthwhile.
For this example: I’ve created a table with details of new functional features to be delivered over a 4 sprint project:
Estimates to develop the automated scripts are inserted (deliverables based on the project plan).
Next – A second table is created to show the manual effort saved when running the automated test scripts:
This also includes regression testing previously developed functional features.
Subtracting the development total from the manual effort total, clearly shows no value in automating.
However, if these functional features were part of a longer running project (i.e BAU), the elaboration results would be completely different.
With the development of the automated scripts for the 10 functional features complete prior to sprint 5, this shows some serious payback:
Armed with this information, it would be easier to get automation buy in from managers and stake holders.
You aso need to consider how many times you'll actually ever run the test as well. Its tempting to try and automate everything, but if it takes 6 hours to automate (depending on how difficult the tool is and how technical you are as a tester), but each manual run is 2 hours, you have to run it more than 3 times to get payback. If you are only planning to run the test exacly as it is 3 times, then dont automate - use the 6 hours to automate something that will need re-running many times over.
A second point is that an automated regression script will never remain static. The data used may need amending (30 minute job?), the validation may change if for example the report output you are checking has an extra field, or the process changes, so you have to amend the script code.
This time needs adding into the tracking spreadsheet so you track all the hours you have ever spent on automation. That way you can show that by spending 2 hours updating the pack, it has kept it current and provided savings for the next n sprints.
Reply
Point 1 - The example above also includes a table for estimating how many hours you expect to save based on the runs. Without this you would never know if automation is worthwhile.
Point 2 - When estimating development you also need to consider maintenance which can be entered in the top table.
Point 3 - With the elaboration out the way, it would be wise to keep a tracking spreadsheet. The one I have devised includes an actual tracking table.