Monday, 23 February 2015

What is Water-Scrum-Fall (and is it inevitable)?

Water-Scrum-Fall is a compound term formed from Waterfall and Scrum. It is used to describe a situation where a company runs their development teams in an agile (scrum) fashion with scrum ceremonies, artifacts and roles, but follows a sequential waterfall or quasi-waterfall methodology both outside of the scrum team and within the scrum team itself.  The outcome for the scrum team is that each sprint becomes a mini-waterfall with lots of documentation input and a set output at the end.  In essence, it's waterfall software delivery using an incremental scrum-style release cycle rather than either a pure "big project" waterfall approach or a pure "agile prioritisation" scrum approach.

This of course is a generalisation; there is no set definition of Water-Scrum-Fall and each company that uses some flavour of it will implement it to fit their own circumstances and foibles.  But the definition above covers the basic truth of the matter.  There is a fuller explanation by Dave West, the man who coined the term, here and a short but interesting piece on the causes and the fact that Water-Scrum-Fall is a reality we have to face by the DevOps evangelist Sanjeev Sharma here.

What's interesting about these articles is that both authors focus on the business reasons for Water-Scrum-Fall, and both argue that a key factor is that the business functions outside of the scrum team are not agile, and therefore continue with their waterfall methodologies.  I don't doubt their analysis, and for the business processes and structures outside of development I'm sure they're correct, but their analysis doesn't explain why development teams continue to produce deliverables in a linear Dev-Test-Doc process within each sprint. They are missing a fundamental point about scrum that means a sequential Water-Scrum-Fall process within the scrum teams themselves is so common:

To be agile you must have a multi-functional scrum team, but multi-functional scrum teams don't exist.

Scrum is a great idea in theory, but so is the star ship Enterprise.  We can't build a functioning star ship without a warp drive, or it's equivalent, and we can't build a functioning scrum team without a multi-functional team, and those basically don't exist.  I've described this problem elsewhere as the unicorn at the heart of scrum, and in essence it boils down to this: It takes time and effort to become competent, and stay competent, at any one of the individual roles in a scrum team; it is almost impossible to become and stay competent at Development AND Testing AND Documentation AND anything else you have in your scrum team.  This is not a question of will, or training budgets, or management buy-in.  It's a question of logistics.  There's a fuller analysis of why there aren't more multi-functional scrum teams here if you want to explore this topic a bit more.

Whilst Waterfall has many flaws, it works because it takes into account the essentially linear nature of development work.  Testing can't complete until Development have completed and Documentation can't complete until Testing have completed.  Different functions can overlap, but testing cannot complete their testing of the software until development have finished development - it's a matter of physics, not choice. Development HAS to finish before testing, testing HAS to finish before documentation, and documentation HAS to finish last. This is not going to change simply by moving to a different methodology because the factor that's missing isn't methodological, it's functional. Or, more precisely, it's multi-functional. 

Water-Scrum-Fall is not caused by a failure to implement scrum correctly or completely.  It is caused by scrum being a theory that requires something that doesn't exist in the real world - a multi-functional team.  

The lack of multi-functional teams means that it is inevitable that scrum teams will move in a linear fashion, because they simply cannot operate in any other way.  If each individual member of the team is single-functional, then the testers must wait for development to finish and writers must wait for testers to finish.  Yes, of course testers and writers can use some time at the beginning of the sprint to write test plans, produce skeleton release notes, and any other tasks they can start or complete before seeing the software, but the meat and drink of their work must happen after development.  A tester cannot fully test and a writer cannot fully document without having the software.  At the end of the process, when testers are testing the last few development items, the developers have nothing to do.  When the writers are documenting the last few work items the developers and testers have nothing to do.

Let us be clear: This is a fundamental problem with the scrum theory.  There are ways of ameliorating this to a certain extent, but you cannot change the sequential nature of Dev-Test-Doc just by changing the methodology to scrum.  So if you don't have multi-functional team - and you almost certainly don't - then a linear in-sprint system is the only realistic way to operate.  This means that Water-Scrum-Fall is an inevitability for most of us, both in and out of the scrum team, and we need to find ways of making that work.