Executive Summary

Acceptance criteria for User Stories is a commonly understood concept. Defining them at the Feature level is less commonly understood, because it is often at a different level of abstraction. In place of the common “Given-When-Then” construct, SAFe recommends “benefit hypotheses.” This Benefit Hypothesis should be augmented with non-functional attributes that apply across the subordinate User Stories. A summary-level “Definition of Done” for Features would then be:

· All subordinate its user stories have all met their Definition of Done

· The summary of User Stories fulfill the benefit hypothesis of the Feature

· All user stories embody or enable the overarching non-functional requirements for the Feature.

Feature Acceptance Criteria

Features are written at a high enough level that they usually don’t have the same type of acceptance criteria that stories have[1]. They have benefit hypotheses that should be validated. For example, I may have a Feature that states I want near-real-time data validation for all transactions involving compensation-sensitive information, including user interactions as well as inter-system transactions. The Benefit Hypothesis[2] would be that data corruption would go down and end-user rework would be reduced. A number of User Stories would be needed to build that benefit.

Feature acceptance for such a benefit hypothesis would have two aspects. One would be “vertical functionality,” in which all subordinate User Stories would have to be in place before the Feature would be acceptable. The Verification view of acceptance ensures that all functionality that was asked for is present, which would be handled by work at the User Story level – pair development, unit testing, integration testing, story demos, and product owner acceptance. A Feature wouldn’t be presented for approval unless all subordinate functional stories meet their definitions of done.

The other aspect is qualitative. Performance, Security, Capacity, Availability, and Reliability are examples of quality attributes. The Validation view of acceptance would check that these are present for the Feature, as well as ensuring that the functionality integrates and “plays well with others” in the target environment. Some “ilities” may apply across an entire system or enterprise; Security, for example, should be constantly re-validated from the task level on up through Product Test. Other “ilities”, such as Performance and Capacity, may be more difficult to validate until an entire Feature is integrated.

Product Managers and Product Owners should consult developers and users of previous manual and automated systems when putting together benefit hypotheses and “ilities.” They should review the acceptance criteria periodically throughout the development work to ensure they did not forget anything critical. Finding an omitted acceptance criteria half-way into the project is painful – but much less painful than finding out near the end of the project.

[1] From a purist standpoint, epics, features, user stories, and tech stories all just stories at different levels of detail. SAFe argues that the “they’re all stories” approach is confusing to Product Owners.

[2] https://www.scaledagileframework.com/features-and-capabilities/

The Benefit hypothesis statement plus any specified “ilities” is enough information for Feature Acceptance Criteria. The typical User Story AC statement (Given <situation> When <action> Then <result>) often doesn’t apply well. If your feature’s acceptance criteria can be easily packaged into Given-When-Then form, then it may be closer to a User Story than to being a Feature.

Agile projects try to not to over-specify Features and Stories. Product Managers and Product Owners don’t want to write a massive, static Systems Requirements Specification document. At the other extreme, however, Product Managers define features with narrow benefit hypotheses; they don’t bother reiterating benefits from existing ways of working that shouldn’t be lost. You can almost imagine their thinking, “Of course the system will still give benefit Y. Anybody knows that.” They assume that all Product Owners and developers will know the history and benefits of prior systems (including manual ones.)

That myopic thinking first of all supposed that the new system will automatically be superior to the one it is replacing, in every way. It also reflects the “feature creature” mentality, in which functionality and advertised features trump everything else. It’s a trap, however.

A Case Study

At one prominent agency, the business sponsors requested that the IT group modernize one of its systems. The Product Owner acted as the sole point of communication with development teams. The primary business hypothesis identified for all the Features to be developed focused on migrating to a hosted Web architecture from the old Mainframe environment. This would save tens to hundreds of thousands of dollars in support costs annually. Additionally, the enhanced system would reference thousands of digitized documents, reducing the agency’s massive paper handling costs.

The Developers correctly transitioned all the functionality from the legacy COBOL code. They successfully transitioned to a “paper free” system, as specified. The Product Owner approved the functionality as it was developed until the finished system was given the “Go Live” approval. It met the “vertical functionality” requirements for all Features.

What the Product Owner failed to do, and prohibited the developers from doing, was observe how the end users performed their daily tasks. Instead of filling out screens of data in linear progression, the end users had previously had a legal reference book open on their desk, the research files arrayed in front of them, and supporting documents on bulletin boards or pinned to their cubicle as they made determinations and filled out data fields in the mainframe system. Their job was a flurry of filling out data fields, checking legacy files, making phone calls, looking up on-line and paper references, sending e-mails with requests for information, saving work until queries were answered, pulling that work back up later, retrieving and replacing some files in physical safes, saving sensitive digital data on secured storage media, meeting colleagues face-to-face over some issues, petitioning judges, and so on. It was frenetic, non-linear, asymmetrical multi-tasking.

With the new system, the end users were forced to navigate from screen to screen to research information and fill out data fields. They were not able to have multiple displays open simultaneously; they had to close out any given window in order to navigate to any other. It was a linear automated flow, heavy in mouse clicks and very low on navigability and usability. Under supervision of a Product Owner, the agile development team created a rigid, waterfall business workflow for this volatile, non-linear process.

This system met User Story acceptance criteria 100%, met the stated set of Feature benefit hypotheses, but trashed an even more important set of benefits the older system had provided. This disaster was an example of 1980’s-style cost cutting, in the name of “business process re-engineering,” revived in the 2000’s. The new system had to be scrapped, and the old mainframe system kept in commission long enough for the agency to field a viable replacement.

The great irony of such situations is agile developers assuming other kinds of work are linear and non-recursive, failing to elicit through observation and end-user inquiry, and using agile development to impose rigid waterfall workflow into the end product.

Summary

Development teams can deliver against stories and features, creating functionality that matches exactly what was stated (verification.) Without appropriate consideration of business value and other qualitative factors, that product may be exactly what was asked for, but unfit for use in the target environment for business reasons or for technical reasons. To ensure the qualities of a Feature are correct, Product Owners should include all the right sources of information, and ensure all the relevant questions have been asked.