Friday, 27 February 2015

Agile as part of Waterfall

When we started developing with Scrum, it was just the development team using Agile, but we were one of many teams still working in a waterfall style (one long cycle with a delivery at the end). This introduced a couple of irregular circumstances when we tried to follow the Scrum process as best we could, for example we always had 2 week Sprints, starting mid week and could fit 3 sprints into the larger delivery process, but we would have a couple of extra days at either end where we couldn't start on the next delivery because the one we had finished (early) hadn't been submitted yet. 


But a couple of concessions didn't mean that we should abandon the whole process altogether. While we wouldn't be able to get all the benefits (we could technically have had a delivery of some sorts at the end of most sprints, but we never did), we still got some and that made it worthwhile. 

The daily stand ups were the most important thing in my view. It allowed the whole team to know what everyone else was doing, and a daily opportunity to help out on problems anyone was having. This always worked well, and there was no conflict to being part of a larger non-agile process. 

One concession we started off with was having both acceptance criteria and a larger formal specification. This did cause some conflicts initially if the two ever got out of sync as we would generally develop to the acceptance criteria, but system tests were done against the spec. As we got better with agile, the acceptance criteria was better defined from backlog meetings so changed less often. I think in this case, if both are to be maintained, it would be better to update the specification after the sprint has finished, so the specification will then match what the acceptance criteria evolved into, avoiding the two going out of sync. 

Our end-of-sprint time was probably most different from the expected baseline. As we were part of a larger process, we didn't have a release. The code was in a state that we could have done a release, but in practice we didn't. Our code area was just one part in a larger product, so even when we did submit code for a release, it went through a full integration process first. We started off doing a demo for most sprints, but the participants and frequency changed over time. In the end we did a demo every two releases (roughly 6 sprints worth of features) to the support team, so they knew how what had been added worked and could ask any questions about it in advance. The component we were developing was the interface into the product as a whole, so I think a demo to any internal customers would have been useful in addition to just the support team. 

Sprint Retrospectives were meetings that I always found useful and we would discuss a lot in them to improve our ways of working. We would always try to get everyone to think of at least one thing that went well (so we could try and replicate it again) and one thing that could have gone better (to try and prevent it happening again). At some point we stopped doing them and didn't start them up again apart from the occasional one-off, which I think was a loss. We did run them for a couple of years, in which time we would have ideally sorted out most (or at least the major) problems. 

Towards the end of my time in the role, the team size was smaller than when we started, and we were essentially doing a 6 week sprint of a mix of Scrum and Kanban, which I have written about before. We used the Scrum practices for stories with acceptance criteria, only taking about half of the velocity for the team, with the other half used for bug fixes using Kanban. 

In the end, working with small Agile developments inside a larger Waterfall style model did work out. The Agile process had to be a lot more flexible than the Waterfall style one in order to fit, but from a high level perspective, our development was only one part of the Waterfall, so we did have the flexibility to do it in a manner that was productive for us, so long as it didn't interfere with the larger Waterfall (which only cared that the work got done, not how it got there). There was still a formal system test afterwards, so while any new features ended up getting tested twice, this was a good thing as we had a much lower rate of new bugs. The grooming meetings were the most valuable part of the whole hybrid mix. Initially it meant we were updating the specification while developing (which is a previous step in Waterfall), however as we finalised a larger backlog we were making high level estimates on features before they had been specified, and because of this less mid-development changes to the specification was needed. 

In summary, running the development part of Waterfall with an Agile methodology can work, so long as you are not too rigid with the enforcement of the Agile method. This may mean you don't get all of the benefit from the Agile process that you could do, but if the development step would improve using an Agile method, the benefit you do get will likely outweigh the alternative of not doing it.