Automation can be a tricky thing – what should be automated? How do you know? Many places dream of having everything automated, and no manual testing at all, but automating just to automate isn’t actually a good thing. And, I personally believe in a mixture of automation and manual testing being the best way to tackle testing, regardless of title or position at a company. There is such a thing as too much, or wasted, automation.

• Repetitive tasks
  • If you’re going to have to test the same thing over and over, then it’s a good candidate for automation. A good example for this are login flows – you constantly have to login (super repetitive task), and those always need to pass.
• Basic Smoke Tests
  • If you’d be embarrassed if the feature didn’t work a certain way and it’s used enough, automate it. For me, if running a smoke test suite would have me repeating this step more than 2 or 3 times I want it automated. Or, if this whole smoke test is something I’m running more than just as a one off, I want as much of it to be automated as possible.
• Time Savers
  • There are some paths that you don’t test all the time, maybe just once a month or so, but, when you do have to test them they take you forever. Automate that crap – such a huge timesaver in the end.
• Happy Paths/Things a User Does Every Day
  • What’s the primary purpose/function of your application? What paths does a user take to navigate said application? If a user has to do something every single day, like login, search, click a button, etc you should make sure to have automated tests to account for those actions. If something that is used everyday for basic functionality breaks that’s a big deal, and a blocker for the user.

Ok, so we know what we should automate, but what shouldn’t we automate?

• Scenarios That You Know Will be Flaky
  • An example of this could be a feature that sends an email. Email is tricky, and you never know how long it will take for an email to be received. Because of that, if you automated this functionality you’d probably get a lot of failed tests, just based on timing, not the functionality actually being broken.
• Features That Are Still in Flux and Constantly Changing
  • I’m currently at a startup where everything is flux, and as soon as I automate something in the UI it changes, leaving my tests in a constant state of brokenness. This isn’t helping anyone, so rather than continue down the path of UI automation, I’m waiting until the product is more stable, and focusing my efforts elsewhere.
• Features that No One Cares About
  • This feels pretty self explanatory, but if it’s a feature that you no longer support, or that no one actually uses, it isn’t a great candidate for automation as it’s not benefiting anyone.
• Weird Edge Cases
  • Those random edge cases that require you to do something crazy, like hit space, shift and t at the same time to get the page to glitch out, do not need to be automated. Just, no.
• One Off Bugs You’re Confident You’ll Never Encounter Again
  • If it’s a true one off, that means it’s not at all likely to happen again. As such, log the bug that you found, but don’t invest time in automating it.

My best rule of thumb on this – if you aren’t doing the same action multiple times, don’t automate it. And, if things are changing so quickly that you’re automation will not be relevant or up to date in a week, then you might be jumping in too early, and you should be looking at other ways to contribute, like helping with unit tests, or doing some API automation, etc.