When is test automation not test automation?

We’ve spent the last week or so scruitinising our smoke tests. Currently our smoke suite is probably best described as primitive as the tests drive the UI; this heavy reliance on the front-end means we’re only testing applications and services indirectly. As such these tests will only ever be as fast as the browser so I’ve been spending time assessing how we can improve them. We know what we want to test, but are questioning how. Naturally, these tests should be automated in one way or another.

Test automation bears a lot for discussion. Automated vs manual testing, whether you should be automating at all, the maintenance of tests, which tools to use and the return on investment are all topics that have been discussed thoroughly for years.

Having spent more time thinking about the how for these smoke tests, I  now cringe when reflecting back at past projects that involved automation. Those woeful days in the ‘end of cycle’ testing era when expensive UI automation tools were thought to be the answer to all of life’s problems. Datacentre move? Automate all your manual tests! Regression deck? Automate everything!

Since the wall between testers and developers has been brought down it’s easier to look at something and work out how to test it. A favourable alternative for a tester rather than hearing about something then thinking about how to test it using only a specific tool and only from the front end. These days working with a development team practicing behaviour-driven development means, from a testing perspective, that I gain confidence in the application sooner. As a result, further testing efforts don’t overlap with what was covered on a unit level and I’m able to invest more time in ‘ility’ and exploratory testing – the agile testing dream.

So, when is test automation not test automation?

When it’s just UI manipulation and the thing that actually needs to be tested is burried away on another tier.

Although it sounds obvious to test specific logic in isolation and on a unit/integration level, from what I’ve seen on automation projects testers don’t always had the opportunity to do so. This is a probable cause to the test-it-all-using-the-front-end mentally and the notation that test automation only involves manipulating the front end.

What about those smoke tests?

We’ve pulled tests out and automated them on their rightful tier (same test, different how), with the use of integration tests that run in fractions of a second. All possible with the help of a toolsmith, of course.


0 Responses to “When is test automation not test automation?”

  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: