Thursday, November 21, 2013

Software delivery organisational anti-patterns

It's good to have some electrical engineering background before reading this post ;P

I am confused every time I am asked what is the best way to manage requirements or which one is better: use cases or user stories, or how to ensure software quality by testing or how to evolve architecture properly, or does the *DD works, which design patterns are most wanted, what is better: agile or RUP. These are tricky questions, especially asked a consultant as I am.

Let's me try to explain what is a tricky part in these questions.

From IT point of view, the only goal of IT-Business collaboration is software delivery and nothing more. I like this term "software delivery" because it simply explains what we are responsible for. To achieve this goal we developed many tools which makes our work more effective. These tools address some difficulties we faced with when we collaborate with business people.

It is very important to notice these tools (show on the figure above) are not either-or dilemmas. You take whatever works and fits to a project context. Please read carefully at least:
  • Writing Effective Use Cases by Alistair Cockburn
  • Introducing BDD by Dan North
  • Decisions, decisions by Dan North
  • Specification By Example by Gojko Adzic
  • Domain Driven Design by Eric Evans
  • The Object Primer or by Scott Ambler

and you will see the tools are not competitors. Use stories are not compete with use cases, TDD is not competes with test-last approach, Scrum is not competes with free-style processes, DDD is not competes with simple anemic model, testing is not competes with test-free approach (really).

There is no "better", there is no "the best". Every single tools covers some aspect of IT-Business collaboration on software delivery. And everyone causes some benefits and consequences. But, you are responsible to choose a tool according to the project context. So yes, you should familiarize yourself with all of them.

The Source of All Evil

Problems start when one want to build a business around software delivery.
To scale the business one have to scale software delivery process to be more efficient.
So, one takes some of the tools and tries to compose The Process, for example: Analysis -> Architecting -> Implementing -> Testing -> Maintaining. So, tools becomes The Process steps and one builds teams around this steps.

You may say: ok it's old-school waterfall process. But, even organisations declaring agile approach have at least Analysis and Testing Departments. The rule is: the larger organisation is the more presented solution is applied. Where is a trick?

Software delivery is atomic. The only responsibility is to deliver software that is satisfying business needs and that's all. But scaling a software delivery we split this atomic thing into steps in a sequence.
We assume we may also split the Responsibility into small responsibilities of every single step. Moreover, we assume that Responsibility of whole software delivery is the superposition of the small responsibilities. In general it is not true.

You know, organisations where software is developed by programmers and tested by testers I may observe a lot of problems with a code quality and tension between Software Development Software Quality Departaments. Why? Because software developers are focused on their own responsibility and testers are focused on their own one.

Software Developers assume that testers will find all bugs and they are less motivated to test their own code. As a result tester have more and more work with bugs. And this a vicious circle.

One more example. Organisations where analysis is done by analysts (God, I'm trembling ever time I hear "Project Management Center") and development is done by developers, mostly have problems with requirements quality. It is happen that people don't know how to implement a requirement.

I think that problems listed above were caused by splitting what is unsplittable by the definition. Software delivery is unsplittable, it is atomic. We may use all tools to deliver better, faster or simpler. But we cannot split that tools into steps in a sequence. It doesn't work in that way. Once again: problems were not caused by the people or lack of competences. Problems were caused by the organisational anti-pattern.

Decoupled teams

Better idea is using tools around software delivery without splitting them.
Do you see the difference? Regardless of tools used the responsibility stays constant.
Ok, that was a metaphor but what about real world solution.
Solution is many stable Development Teams. A Team is responsible for delivering a complete working product. Moreover, there is no any dependencies between teams. A team is atomic.

Yes, I know there is more question than answers. How large a product should be? How to cooperate with other teams? How to build and develop a team? And so on, and so on.

The truth is we are in alpha-phase if about scaling agile teams on large projects and organisations. Some inspiration you may find in Disciplined Agile Delivery by Scott Ambler. Absolutely great case study is Scaling Agile @ Spotify. I believe the answer is out there and it will come soon.