When should you automate your software tests? | Agilitest blog

What are the usual answers when it comes to implementing test automation?

Automatic tests should be set up as soon as the features (or US) they cover are validated

Tests should be automated before they are even run for the first time

This view of automation is fairly traditional (often linked to the V-cycle) because automation comes quite late.

Test automation needs to be done when you have time

Automate when the right tool for automation has been found

In this case, the interest in automating its tests is clear, but the tool that meets the need has not yet been identified.

Automation should be carried out when human resources are available

Do not automate

To tell the truth, none of these answers is wrong because each can be totally valid depending on its context. I obviously have a preference, from my Agile experience, for automation during the sprint or even upstream! Nevertheless, I have been led to postpone test automation according to specific contexts. In fact, if I had to give a generic answer to the question “when should you automate your tests?”, it would be:

As soon as possible!

This answer assumes that automation is an investment and that the earlier it is done, the better the return on that investment. Similarly, the use of the word “possible” implies that the moment when it becomes interesting does not depend solely on one’s will, but on the context!

So another question arises:

Depending on my context, when and how should I automate tests?

Tests must be automated as soon as features (or US) are validated and covered

In order to achieve regular automation of each feature before its final validation, it is preferable to add this criterion in the Definition of Done. But also and above all, identify as soon as possible the tests to be automated and identify their environment and data requirements. It is also essential to integrate these tests into automated test campaigns as soon as the functionality is validated, while ensuring that existing regression tests are maintained.

Automate tests before they are run for the first time

With a high degree of rigor, it is then possible to use these BDD tests to have automated tests even before the start of development. These tests are then designed in BDD and, as with TDD, fail before the functionality is even delivered. It is then possible to run these automated tests directly to participate in the validation of the User Story.

You should automate when the software or block under test is stable

This is particularly true in a V-cycle. But it may also be appropriate in Agile when many problems need to be solved.

Automation should then be carried out as soon as a permanent solution emerges.

You should automate when you have the time

I personally have already had to make this choice with the decision not to automate for the first 3 months because an MVP had to be ready for this deadline and we did not have the time to automate the tests (other temporary concessions had also been made).

You have to automate when the right automation tool has been found

In this case, it is preferable to carry out a market study followed by a POC in order to determine which tool will best suit our technical, human and budgetary context. The tool (and its choice) should not be a hindrance and should not waste the team’s time.

We need to automate with the right people

These problems are frequent, and this is why a tool like Agilitest was developed. Indeed, Agilitest makes test automation accessible to non-technical profiles, leaving test professionals to deal with the automation of test execution.

You don’t have to automate

In this case, it is necessary to spend a lot of time on manual tests.

Finally, why are you thinking so much about test automation? What is the benefit of test automation and why automate “as soon as possible”?

Why automate tests as soon as possible?

Beyond the simple financial aspect, automating tests as early as possible enables regression tests to be integrated, to be executed more often and therefore to test earlier and more frequently! Early detection of the most significant anomalies saves a lot of time and also improves the morale of the software development teams.

Finally, in the case where tests are automated before development begins, the tests are shared with the developer before the developer even starts building the functionality. This helps the developer to better understand the requirement, but also to ensure the validity of his work beforehand. We then benefit from all the advantages of automation but also from BDD with real collaboration.


No strings attached. Only good tests.
BONUS: the tests scenarios you create during your trial period can be replayed in ATS, our Open-Source backbone, for free and forever.

Originally published on Agilitest’ blog.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store



Codeless functional testing at scale is now a reality.