Requirements in Agile

[jbeatty]

Roger posted a great comment on a blog post (http://requirements.seilevel.com/blo...ot-pursue.html) that I wanted to have further discussion around. I'd like to hear what people's experiences are at doing requirements gathering in agile? What worked, what didn't?

One of the common arguments I've heard for agile is that requirements change, so do not bother to define them up front. I have to ask, do requirements really change over the course of a project? I'm going to go ahead and throw out the idea that often requirements change over the course of a project because they were not elicited and written correctly, not because the actual users are changing their minds.

I'm also not proposing a strict waterfall approach either, where you must have every last detailed requirement written before design and development can begin. There is a middle ground where the requirements are stabilizing and the dev team should begin, because rework on their part will be minimal at that point. Perhaps there is a middle ground accepted in agile that I've just not seen.

[JGrapes]

One of the common arguments I've heard for agile is that requirements change, so do not bother to define them up front. I have to ask, do requirements really change over the course of a project? I'm going to go ahead and throw out the idea that often requirements change over the course of a project because they were not elicited and written correctly, not because the actual users are changing their minds.

I agree here. And, I think that when people say you need to just develop the software to find out what the users want, the question we're really addressing is "What is the most cost-effective way to gather requirements?"

If programmers are developing requirements and programming is how they think, model, and explore requirements, then that may actually be the most effective route for them. But, from an organizational point of view, is this efficient? Are programmers the right people to gather requirements? Is programming the quickest way requirements gathering can be done? I believe a good requirements analyst using their tools can gather the requirements much more efficiently. (And, a good programmer can implement them much more efficiently than a requirements analyst who knows a bit of <fill in language of the day here>.)

I don't think "...requirement change, so do not bother..." is part of the principles behind the agile manifesto (http://agilemanifesto.org/principles.html), but a tenant of some of the development methodologies that claim to have grown out of it.

The principle "Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage." notes that you must allow for change, but doesn't state that all requirements change.

They also state that "The most efficient and effective method of conveying information to and within a development team is face-to-face conversation". I both agree and disagree with this statement. This is a very efficient way to receive information. I like to be able to ask questions when receiving information. However, if you are distributing the information to many people (15, 50, 500...), I don't think this is an efficient way to distribute information or that the distribution of the information will be consistent (assumption: you will not always be able to talk to all of these people at the same time). This is where documentation comes in. Also, if I'm receiving information which contains a lot of very specific details, I'd prefer to get it in writing. The conversation is great for concepts, big pictures, and exploring open questions. This principle is often used to justify "don't write requirements down". But, it doesn't say that.

[LucyBA]

My one experience thus far with Agile was with a development team doing 2 week iterations about 6 months ago. The BA's did everything we could to gather the requirements before developers started into their cycles, since we worried it would be chaos to write good requirements once they started developing.

The users and the architects were paranoid about using this method. They feared that the overall architecture of the system would be wrong if the requirements weren't gathered first.

The developers wanted lots of details - like what exactly should go in the UI and where, but the screen designers could not design until they had requirements. The developers wanted to know exactly what was required in so much detail, but with 10 developers and 2 BA's there was no way we could keep up on the requirements in 2 week iterations. It was a frustrating experience for the BA's and developers.

As I learn more in different styles of projects, I may have been able to do it better another time around. But in the end the project was a pretty big failure.

[achen]

My one experience thus far with Agile was with a development team doing 2 week iterations about 6 months ago. The BA's did everything we could to gather the requirements before developers started into their cycles, since we worried it would be chaos to write good requirements once they started developing.

The users and the architects were paranoid about using this method. They feared that the overall architecture of the system would be wrong if the requirements weren't gathered first.

The developers wanted lots of details - like what exactly should go in the UI and where, but the screen designers could not design until they had requirements. The developers wanted to know exactly what was required in so much detail, but with 10 developers and 2 BA's there was no way we could keep up on the requirements in 2 week iterations. It was a frustrating experience for the BA's and developers.

As I learn more in different styles of projects, I may have been able to do it better another time around. But in the end the project was a pretty big failure.

Agile is just a subset of iterative programming. I dont think that there is anything in agile that absolutely requires 2 week iterations (I have seen 2 month iterations). Two week iterations seem closer to extreme programming. I really favor 4 month releases with an up front planning phase that overlaps with design and even partially with development.

[phyfe]

I was in an agile shop and felt that the project was successful.

what worked.

• Developers and analyst in the same room (called the pit). Lots of discussion and collaboration.
• Quick iterations. (2 weeks, 4 weeks or 6 weeks). Forced us to break tasks down so a developer wasn't able to get muddled in something too complex.
• After each iteration we presented the product to the client. (And yes there were changes)
• Pair programming - it works!

what did no work

• Complete disconnect from the requirements themselves. Solely relied on Use Cases and Story cards.
• Use Cases and Story cards over documented prior to hand off to developers. Confusing, too many questions, lots of changes.
• Release management was all over the place. No clear ability to say what is in the next release.

[Jerry]

what did no work

• Use Cases and Story cards over documented prior to hand off to developers. Confusing, too many questions, lots of changes

I would love to hear a little more about what exactly you mean by "over documented" and how you would have liked to have seen it done differently.

[phyfe]

I would love to hear a little more about what exactly you mean by "over documented" and how you would have liked to have seen it done differently.

Ah, good question. Hard to answer and it may well be a personal opinion. But I think it's an important dialog to have. First of all I feel that there needs to be a clear distinction between what I think of as a "Requirement" and everything that comes after such as Use Cases, Story Cards etc. The initial requirement in my mind should be simple and concise and something the whole team refers to continually. After that it's anything goes. The reason your query is so interesting is that this question is certainly one I grapple with often. After all, who's to say that there is such a thing as too much documentation. The more the better right? But the more I read and the more projects I work on the more I see there is a point where too much documentation becomes detrimental.

So what is too much. Well, it depends. It depends on the team, the process, the client, the tool etc. For example I was on a project where we had the requirements document totally written before anything started. From there Use Cases were developed iteratively. As well the development team was required to draft up technical documents on how they were going to implement the said Use Case. In the end the original requirements doc was forgotten, the technical specs where quickly out of date and the Use Cases were read primarily by the Analyst and the Project Manager. In the end the only reason I feel the project worked was because it was a good team who talked a lot.

Then on the last project I was on we had a very dynamic and collaborative environment that worked very well for the developers but the written use cases where typically 5 to 30 pages long with diagrams, rules, flows, screen shots. Even after all of that, the developer would within an hour of reading this document call up the analyst and say; "I'm not sure I understand what is meant in section so and so" or "What if I did a drop down here instead of a check box".

I guess in summary I feel that too much documentation (over 3 pages) makes for a too ridged environment while too little (less than one page) leaves too much ambiguity. And I guess that the other thing I'm kind of realizing now is that in every situation I've been in, the success of the project came down to the whole team talking not necessarily the documentation. However it's always important to have somewhere to start, a requirement, and then from there depending on the team, different members can associate to the requirement either additional documentation, diagrams, images etc, that are meaningful to them or the team.

Whew, sorry, kind of long, looks like I'm not following my own advice.

[Jerry]

Thanks for elaborating phyfe!

[dwwright99]

Agile falls short, says author

Software development concept hasn’t blossomed: McConnell
By: Paul Krill(31 Mar 2006)

From Computerworld Canada: www.itworldcanada.com
The article is at http://www.itworldcanada.com//Pages/...0-9fa5a564512e
but you might have to register to read it.

Steve McConnell lists his worst ideas of Agile, my favourites being:

•Requirements are always changing.”[The] single most common source of changing requirements [is] requirements that were not significantly investigated in the first place,” said McConnell.

• Requirements can be gathered or they just drop out of the sky like manna from Heaven.

• Entrepreneurial companies cannot be afraid of risk.

• One single development approach will work best for all projects.

Mr. McConnell is a published author on software topics; while I don't know what his original views of Agile might have been, he has certainly laid a shot over the bow of agile proponents.

Comments, anyone?

[phyfe]

I can't help but wonder if Steve McConnell has read the Agile Principles.

•Requirements are always changing.”[The] single most common source of changing requirements [is] requirements that were not significantly investigated in the first place,” said McConnell.

In Principle #2 they say "Welcome changing requirements", To me this is not about putting less weight on good requirements gathering as much as it's about being prepared for change. I think it is dangerous to associate a change to poor initial requirements gathering.

• Requirements can be gathered or they just drop out of the sky like manna from Heaven.

I don't recall reading anywhere in agile that requirements just happen magically. There is a great diagram on page 3 of this pdf which is chapter 2 of Craig Larman's book. I personally feel it demonstrates a much more realistic view of a project from start to finish.

• Entrepreneurial companies cannot be afraid of risk.

Try telling that to someone who quits their job to build a product in their garage. Entrepreneurialism is all about risk. I can only assume that what he is referring to here is agile's leaning towards a more creative and self managing process. Many cooperations may see this as loosing control and therefore risky.
Worth reading - Agile Manifesto

• One single development approach will work best for all projects.

Agile is certainly not a single development approach.
Scrum
XP
Crystal Methods
to name a few. Here are more