On the friction between agile and traditional approaches in software delivery
I’m an Agile pragmatist.
What do I mean by that? I find that for practically every piece of software I’ve ever written, or seen written, agile practices will deliver a solution that is a better fit for the customers needs, with something valuable in a shorter timeframe, and with happier developers than a ‘traditional’ Big Upfront Design (BUD) approach, e.g. waterfall.
But, and it’s an important but, I’m not ruling out that there’s a class of problems where BUD has it’s place.
What is that place?
To answer that, let’s take a side to to Value Stream Mapping Simon Wardley at QCon 2015 gave a talk that touched on this very topic. An introduction to value stream mapping, I wish you could have seen it, or that they had made the full 40 minute talk public, you will have to make do with this 13 minute flash talk from OSCON 2014. It managed to be funny, insightful, and enlightening in just one talk.
The evolution of technologies
One of the key takeaways for me from that talk was that all technologies go through an evolutionary process. Overtime they will move from Genesis, through custom built to product, and eventually, to commodity. He uses the example of light sources:
We started out with a candle - eventually we get Edison’s light bulb, but that requires custom installers, and eventually we get to LED downlighting that you can install yourself.
We also had it with electricity itself, 3 thousand years ago the Egyptians had begun exploring electricity with Seleucian jar type things (a myth, apparently), then with Tesla and Edison we get the shift to commodity electricity to the wall through Westinghouse.
The point he made that I found so compelling was that on the left, in emergent technology, agile fits brilliantly – we’re dealing with the unknowns, the unknown-unknowns – but over on the right, as things become commoditized, we have more certainty, predictions/estimations are more reliable. Those traditional project management approaches like Waterfall and PRINCE2 make sense in this space. All the steps are known, and it’s just a matter of following them until you’re done.
10 years ago Jeff Atwood was blogging on this very topic in “Is software Development like Manufacturing”. While I’m a huge fan of the realisations and improvements that have come LEAN, misapplying project management from manufacturing to discovery/engineering pursuits on the left of the graph is a long-standing problem.
- Variability is the enemy in manufacturing; in software, it’s the reason we get up in the morning. Every worthwhile software development project is a custom one-off job for a unique problem.
- Requirements are the bread and butter of manufacturing; in software, we rarely have meaningful requirements. Even if we do, the only measure of success that matters is whether our solution solves the customer’s shifting idea of what their problem is.
– Jeff Atwood
Patrick Mockridge — CEO of engzig.com — touches on the same topic though a little more tongue-in-cheek.
I think a lot of discomfort in software development comes from the friction in the interaction between these two worlds. In the traditional camp you’ve got business managers, possibly from an operations background, used to performing actions such as getting a trade to settled. And in the other you’ve got ideological ‘agile’ developers who can be resistant to estimating anything.
Frequently the business managers become project managers, or sponsors of projects to replace some, or all of the process they’re used to performing. Never mind the potential for conflict-of-interest (“we’re automating their jobs!”), if you’re lucky enough to have one that buys into the development effort, in my experience it’s likely that they think software development fits the mould of widget stamping. Because they’re used to performing to measurable targets, and motivating their people to achieve them, they think software development can be hurried the same way.
Simon Wardley shows us that some activities we class as Software Development may have evolved to occupy that product or commodity side of the chart, these are the places where the ‘software laborer’ can exist. It’s the rote, copy/paste expansion of functionality on systems were those functions haven’t been made so simple the user can click themselves. However, these are unlikely to be positions we want continue to occupy if we want strategic business advantage.
This is where I believe it’s important to do cost/benefit analysis of any improvement, and apply it to a value stream (different to a value chain map!) — Checkout the amazing work done by GDS for the Criminal Justice System — But having done that analysis, don’t consider your plan set in stone.
The other important thing to realise is that the distance between genesis and commodity can be huge. I’ve seen the difficulty/amount of work involved in bridging that gap under-estimated time and again. A solution to that is to accept it’s not necessary to do that bridging yourself. Choose carefully and open-source where you can to build a community and the evolution will occur given time. Spend your effort on building the novel.
Lance Paine 23 March 2016