Recently I hosted a DCG webinar on user stories. A few days prior, I emailed the current list of registered attendees and asked that they send any questions in advance of the webinar (I addressed additional questions at the end of the webinar too, of course). I received a number of fantastic questions and would like to share one in particular that struck me as interesting.
Which is a better metric for project planning:
- User stories
- Local self-made proxies
- Functional points
- Any other options?
Given the topic of the webinar (a primer on user stories), my answer focused specifically on whether story points are the best metric for project planning.
Size is one of the predictors of how much work will be required to deliver a project. Assuming all project attributes, with the exception of size, stay the same, a larger project will require more effort to complete than a smaller project. Therefore, knowing size is an important factor in answering questions like “How long will this take?” or “How much will this project cost?." While these are questions fraught with dangers, they are always asked. If you have to compete for work against other companies or consultants, they are generally difficult to avoid. While not a perfect analogy, I do not know a person that builds or is involved in building a home that can’t answer that question (on either side of the transaction) - the question is a practical one.
Selecting which metric to use to plan the project depends on the type of project or program and whether you are an internal or external provider (i.e. whether you have to compete for work). Said a different way: as all good consultants know, the answer is, "It depends."
User stories are very useful, both for release planning and iteration planning, in projects that are being done with one or a small number of stable teams. The stability of the teams is important so that the team can develop a common frame of reference for applying story points. When teams are unable to develop a common frame of reference (or need to redevelop the frame of reference due to changes in the team), their application of story points will vary widely. A feature that in sprint 1 might have been 5 story points, might then be 11 in sprint 3.
While this might not seem to be a big shift, the variability of how the team perceives size will also be exhibited in the team’s velocity. Velocity is used in release planning and iteration planning. The higher degree of variability in the team’s performance from sprint to sprint, the less predictive. If performance measured in story points (velocity) is highly variable, then story points will be less useful for project planning. Simply put, if you struggle to remember who is on your team on a day-to-day basis, story points are not going to be very valuable.
In addition, external providers generally have strong contractual incentives to deliver based on a set of requirements in a statement of work, RFP or some other binding document. While contracts can (and should) be tailored to address how Agile will manage the flow of work through a dynamic backlog, most are not, and until accounting, purchasing, and legal are brought into the world of Agile, contracts will be difficult. For example, outsourcing contracts often include performance expectations. These expectations need to be observable, understandable and independently measureable in order to be binding and to build trust. Relative measures, like story points, fail on this point. Story points, as I've said before, are also not useful for benchmarking.
Story points are a team-based mechanism for planning sprints and releases. Teams with a rotating door for membership or projects that have specific contractual performance stipulations need to use more formal sizing tools for planning.
VP of Consulting & Agile Practice Lead