Why checking an AC isn’t enough

I’ve recently had the realisation that even within agile environments testers and developers can take the view that we can check our Acceptance Criteria (ACs) to “pass” a story. Working within a number of agile engineering teams I wanted to spread the view that testing an AC doesn’t mean saying “yes it does that” instead it means “Okay, here’s how it does that”.

What is an AC?

Paraphrased from Google: An acceptance criteria is a predefined statement of behaviour or functionality that we want to meet in order to achieve a user story. It’s a short statement telling me what I want from this feature; similar to what I’d see from a requirement of a specification. However the context of being in an agile environment means that there’s a big implicit difference: scope. 

In the old days of waterfall requirements, designs and specifications were written out to cover everything; this means that each requirement can be taken as a whole complete ask with little need to expand upon it (as the rest of the document will do this for us). By comparison, working in agile means we’re working with more uncertainty and using ACs that are open to interpretation; we don’t explicitly call out everything we need to cover. This means we have a lot of implicit asks that an AC might have that we need to tease out through our testing.

Implicit asks

Let’s take a look at an example user story and the explicit, implicit and assumed requirements:

When a user is on the login page and logs in with a valid password they will be taken to the home page.

Explicit requirements:

1. There’s a login page
2. A user can log in
3. Passwords will be validated
4. There’s a home page
5. The user will be displayed the home page automatically

Implicit requirements:

1. A user cannot log in with an invalid password
2. There will be a UI that allows for the user to log in (fields)
3. The user cannot access the home page without a valid password

Assumed requirements:

1. An error should be displayed for an invalid password
2. There will be a user account
3. The user account will be stored with a password
4. More than one user can have an account
5. A user can set a password
6. A password can change
7. A password can follow rules to make it more secure (length and characters)
8. A user has to log in each time they access the site (from a new session)
9. The UI will be accessible
10. The user will be using a web browser to access this
11. The user can access the site from a PC, mac or mobile device

Unknowns:

1. How fast does this login need to take place
2. Should the log in be secure (Hide passwords? Automatic log out?Secure web traffic)
3. How should the UI look?
4. How many users will use this at once?
5. What languages will this be displayed in?
6. What happens if the services go down?

By breaking out some of the additional non-explicit requirements we see that the explicit happy path is only the tip of the iceberg in terms of this AC. If we just ran tests to check that “yes I can do this” we wouldn’t look at the implicit, assumed and unknown so would be missing out on discovering a lot more information about the quality of what we’ve implemented.

Fig 1: The epic iceberg of explicitness where most of it is assumed to be below the water.

Conclusion

Product teams need information to know how what they’ve implemented meets the AC, not just that the happy path works. This information helps us to refine our product, make quality decisions and make the product better. So don’t just check your ACs, explore them and get information about more than just the explicit!