Return to site

Step 8: Testing Instead of Planning

· 12 Steps

Business is, or should be, about creating value. That sounds wonderfully true and simple. However, plenty of obstacles get in the way. Three of the most significant are:

  • We often don't search for value, we look to save on cost.
  • We can't truly understand value until we know what customers will buy, enjoy and share (even that is frequently a work-in-progress).
  • We don't test enough.

In this chapter we are going to talk about testing. Most companies rely on planning as a kind of quality control. But plans are increasingly inflexible You cannot be business agile and be a planner. Testing can take up the slack. Creating a good testing culture is entirely in line with the direction business is headed in. Before you think plan, think test.

Of course, we want to talk about testing in the context of business agility. However, much of the smart thinking in this area comes from software so we will be looking at that and drawing out lessons that new developments in testing software have for creating business agility.

In approaching this topic we need to understand that business everywhere is becoming more experimental. There is hardly a business sector not touched by the need for deep change. Experiments imply tests.

So how do we go about being testers? And what should we be testing? The answer to the latter is value. We need to test for value creation. What about the former?

Don't Lean and Agile Methodologies Provide Value Methodologies Anyway?

In important methodologies like Lean and Agile, we believe we have tools to assess value when in reality we do not.

The best these methods can do is tell us where resources are being wasted. They are efficiency methods, not value methods and, importantly, they condition people to think in ways that can be anti-value.

To create value in Lean is to take out cost and to streamline. That creates and perpetuates a culture that priorities only one set of values, waste reduction.

In Agile, value is more elliptical. We know we should build value but it is not clear how. There is nothing in Agile methodology to guide us.

Yet in the wider business community the consensus is that value begins and ends with how customers feel about their experiences of a product or service. The job of value-seeking behaviour at work is to seek those actions that make for a better customer experience.

The reality of often very different. While every lean and agile project out there believes it is more efficient than ever we are often guilty of becoming very efficient at creating little value or maybe even destroying it.

There are many instances when teams work on projects that add very little value, even though they are executed well. The two, efficiency and value creation, do not go hand-in-glove. That point was made to us recently by an executive at the British bank Barclays where for a while teams used throughput as a measure of success - in other words more code or more apps done more quickly.

Doing more can mean doing worse but we still obsess over these volume types of metrics.

Seeking value is actually a multifaceted activity. We've already pointed out that In Lean and Agile there's a real struggle to define value in positive terms. In Lean, value is an absence of waste. In Agile, value appears to be a step along the way to completing a project.

To ensure that the work you commission or execute is adding value to customers, you need a multifaceted approach (see Chapter 6) and you need value-seeking to become an embedded behaviour. It has to be a value in its own right. When it is, you will find that some of your treasured processes become irrelevant.

In particular some of the big metrics start to become a handicap. In software this is true of test-suites but it can be just as true in marketing, where, say, the volume of likes or number of followers a company has is judged as an essential metric. These volume metrics don't always make sense.

Value, quality and testing

When you look at value in a multifaceted way it forces you rethink how you test for quality.

In point of fact, it forces you to do a lot more. You need to reappraise many team roles too. The idea of a "multidisciplinary team" is the wrong term for this as everybody has to become multidisciplinary. It is not a feature of the team but of everyone's role within it. in software that is true of developers and tests (their reels can be interchangeable) and BAs and product owners. We are fast moving towards fungible roles where we can, thankfully, break out of the personal silos created by HR policies.

But back to quality. At the very least you need to know that work is of consistent and good enough quality to deliver value. You'd be surprised how that simple idea gets distorted in many organisations.

Most organisations have a cadence to their work. For example writing up a project proposal for executive approval might be regarded as a three week's long task. In our experience good proposals can be generated in a couple of hours but that doesn't stop a product owner from taking the project off on a three week tour!

This cadence of activity, the "what's expected" approach to innovation is killing companies in the West. it is irrelevant to many Chinese companies who are anyway innovating all the time.

We got talking about the quality problem with Alan Murphy who has a major role to play in defining quality at Paddy Power, one of the stars of DevOps and Flow.

Paddy Power is not an untypical digital company. It has a "brick and mortar" presence on many High streets but the real power of the company lies in its ability to create new digital products on the fly. By the way, we are assuming you are familiar with this ability. Paddy Power manages 7 million prices changes every Saturday afternoon during the soccer season but it is also pushing out new products (in this case bets) all through the day.

Alan Murphy specialises in software quality control and is passionate about it. He seemed like a good person to talk with about the issue of quality. Above everything else, he is passionate about unit testing.

Unit testing is distinct from the traditional software testing regime. But what does testing software matter to marketers or other people in the business?

In software, test suites are usually drawn up at the beginning of a project or a significant sprint. What makes this analogous to other parts of the business is that testers are expected to anticipate everything that might be relevant weeks or months down the road and plan their tests accordingly.

Because, in reality, nobody can do this it leads to a very curious outcome. Many projects fail their acceptance tests and are put through into production anyway!

Our belief in the power to plan is reflected in a whole range of projects that executives commission or buy-in. Even the act of digital transformation is a big plan.

Inn our experience people are bad planners and planning is neither always relevant nor always possible.

As we move towards smaller units of work across the business, then testing in a granular way becomes more relevant. As we move, towards a more experimental enterprise, testing itself becomes a cultural necessity.

It might seem anachronistic to think of value stemming from applying tests to very small units of work, which is what happens with unit testing. You might argue that there is some overkill in testing so much. But new disciplines in software, like DevOps, mean that work has to be reorganised to facilitate speed. Companies grind to a halt when bad work is let through to the public and then has to be reworked, so testing becomes more important and more relevant even to speed.

As with all the other elements of Flow we believe the principles apply to all areas of a company's work. Very often we work alongside colleagues who claim a special degree of expertise and won't be challenged (or put another way, won't be tested). They can be pretty vocal about what they know to be true.

That kind of attitude could come from, say, copywriters who are responsible for generating website content. Their traditional role was to be the creative. Companies relied on their intuitive feel for words and audiences. Now, however, we can test most content to see what type of audience it generates. So while Alan talks about software testing, the lessons are applicable to other digital skill areas.

Managers fail to realise the benefits of unit testing

The big issue for the unit testing advocate is that management generally sees unit testing as a waste of time and an activity that holds up coding (the old throughput mentality).

Managers and other stakeholders perceive unit testing as adding cost but that idea, and the elimination of unit testing, is a completely false economy because you will be breaking something in production later and then you will have to fix it in production.


It is very easy to look efficient by pushing code into production quickly but it will just be coming right back at you as bugs.


There's also a tendency to allocate test development to inexperienced developers, often people who are young and new to the company, who you are asking to write quite complex test suites for acceptance testing. It never, never works and it always ends up needing more development work. When you have broad unit testing coverage you can blaze into production with very small changes and very little blowback.

The Testing Environment

Breaking work down into small units allows for better unit testing.

Unit testing is made possible because Flow teams break work down into very small units. In Paddy Power that meant 3.-5 day units, though now Fin works in 2 day units, and shorter if possible. Mastercard recently revealed that they allow a 12 hour window to respond to micro trends!!

Small units of work are very relevant to DevOps. Because DevOps incentivises getting work right as it goes into production rather than reworking later, you need methods that ensure quality as far upstream as possible. That's what unit testing gives you.

Now imagine the equivalent in marketing. A piece of copy has been signed off at the CMO level but as it goes into the companies different channels it looks as though the test if witty and wonderful in print but is very poorly optimised for SEO, so poorly optimised that the likely knock on effect will be either poor uptake or the need for extra marketing budget for search engine marketing (SEM). Bidding for keywords can be ruinously expensive so getting the text right is materially important for a number of reasons.

This kind of copywriting project needed breaking down into small units of work, just like modern software. It will go into different environments so it needs to be optimised for each nd often it will have to be optimised quickly (see Mastercard above. You have a 12 hour window to create code, text and offer!). We can learn a lot fro our software colleagues.

According to Alan, a unit resting approach creates a unique test-driven environment.

We allied unit testing with acceptance testing and really departed from industry standards when we did that, as well as by getting the developer to code the test. We have a just-in-time discussion of acceptance criteria but very lightweight. We quickly convene the product owner, tester and developer, build up 4 or 5 use cases and ask the simple questions, given x, when y, then z should happen. Does it?

Just-in-time acceptance testing can be applied anywhere. The example we gave above of Mastercard's 12 hour turnaround is ideally suited to rapid just-in-time methods that create, test and deliver work at exceptional speed.

It is only possible when you create small units of work. You can run tests on the unit and get reliable, consistent results and then quickly get acceptance for it in place of designing acceptance tests weeks in advance. The marketing equivalent would be that situation where your work is too tightly coupled to the work of people serving other channels.

J-I-T acceptance testing means you are more likely to be testing work of value. Those 4 or 5 use cases are a checkpoint for work-of-value not just quality and acceptance. And in good organisations the developer takes on a bigger role - querying whether a unit of work actually creates value.

The longer term acceptance testing model forces everybody to stick to a plan regardless of whether or not the plan is creating real value and regardless of changes needed within the plan, but often ignored. In Alan's experience there are often enough occasions where code never passes acceptance testing - sometimes because the tests are too difficult to write! - but nonetheless goes into production.


Redesigning work practices in this way, often on the fly, comes with a cost. It causes a lot of friction as developers need to learn new roles. But so too do testers. And product owners have to become more sensitised to the real-time nature of the work.

The analogy in marketing would be the introduction of search engine marketing and search engine optimisation and then from there, the introduction of social media. Bother of these developments have forced marketers to continuously develop their skills and roles, and to introduce data on performance that simply was not available in the older days. They have become testers of their own performance.

That process could be taken further, faster. Although marketers do more of this with their online content than they used to, we believe there is still a long way to go before they truly pass the Flow test. They need to segment their markets more, broaden and diversify product offerings and create more real-time units of work across the long tail of customers. Just as in software there is a reluctance to see testing as central to all this. Testing, done properly, can replace planning.

One of the biggest inhibitors to unit testing is developer ignorance, Alan says. Developers haven't been exposed to this type of thinking. Testing has been treated as a separate discipline.

So developers need to learn how to write and code automated tests. Testers need to learn to operate with developers in writing smaller, automated unit tests. But what about product owners.

According to Alan that role can change almost on a weekly basis. The best product owners become embedded within development teams and have a voracious appetite for networking the developer team with stakeholders, constantly seeking out points of view and liaising when the business objective needs to change, rather than the development track. The product owner becomes pivotal but not in any grandiose sense. She becomes pivotal because of her energy and skills in identifying the right balance between changes in business objectives and changes to units of work.


Some of the grief that comes with change is also caused by confusion in those upstream roles. Product owners need to reappraise how they work but so too do project managers and product managers. The most difficult work structure for Flow practitioners is the project.

Projects get green lights from far off, they get a budget and they get momentum and that makes them almost like a missile when they get into the IT sphere, unstoppable. They resist all the logic of small units of work and unit testing because they have been mandated to work.

The Benefits of Unit Testing

Here is the summary of our conversation with Alan, mixed in with some of our own observations about quality, testing and value:

  1. Traditional testing suites bite off more than they can chew, as do traditional plans of all kins. They test a matrix of code that is in reality is too complex to test. Very often code will not pass acceptance tests and that means projects go forward already compromised.
  2. Code often gets pushed into production too early and conflict between development and production goes up another notch. 
  3. This is partly because, traditionally, work was considered "done", in agile terms, too far up the value chain when we don't know if it will be adding value or if it functioned properly in the wider project. Work is not done until you have evidence it has value and that usually means once a customer has used it and provide feedback.
  4. Unit testing is an answer to software quality problems and hence to keeping work flowing instead of suffering blowbacks.
  5. At the unit test level it can also be coupled to just-in-time acceptance tests that are a rough and ready consensus that it meets our current understanding of requirements.
  6. Unit testing is what really makes DevOps work; it's what allows you to blaze code into production with minimal changes.
  7. Units of work are the smallest unit you can break work down to (see our article on units of work, here).
  8. Good practice should involve a broad coverage of unit testing.
  9. It should also involve developers challenging product owners about the value of any increment of work; given a unit of work as a responsibility a developer can ask cogent questions about its value to business outcomes.
  10. That means everyone if focused on value.
  11. It also forces a continuous reevaluation of key roles like product owner, product manager, project manager.
  12. Software quality tests have to be coded by the developer. They can be designed along with a tester but the developer needs to take the responsibility of coding them.

So those are the key points of Alan's experience.

Applying The Principles of Unit Testing Elsewhere

How can these experiences and thoughts be applied outside IT? We already touched on that above with respect to content and SEO. But there are numerous other ways. In Flow, the book, we pointed out the temptation on the part of many executives to buy big projects either from external vendors or from internal champions. The problem of big projects is that they push value too far into the future, far enough that sometimes the original champions have already left.

And they rely on exemplary use-cases - such as company A did this and saved $10 million. Very little in those use-cases is tested or testable in the way we are proposing. That's why in the book we go into detail about how Flow can be used to develop and test parallel projects that are smaller, lightweight commitments that are capable of delivering strong results.

In addition nobody should forget that everything in business is trending towards small. In the financial sector, as an example, banks are looking to Fintech startups for inspiration on what to change and how.

Software systems are trending towards microservices that communicate with each other.

Marketing is no longer about TV ads, print and outdoor. It has to take into account Snapchat, as well as Facebook and Google, Pinterest, bloggers and influencers. These are all smaller initiatives compared with anything from the past and they carry just as much scope for testing at a small scale as software does.

We are even seeing the most dramatic system changes taking place because of the skills and passions of small companies. In the financial space the ICO or Initial Coin Offering has changed the way funding is raised for startups. This is. product of small companies but it is changing how we view value-creation.

In insurance, we have given the example of drones. Drones are a game-changer for insurers both as a forensic device, allowing them to take an aerial view of any damage, often in locations that assessors cannot properly reach, and also in disaster response.

The use of drones introduces new system requirements such as drone fleet management. Unlike car and van fleet management, drone fleet management is a real time interactive activity. Somebody has to steer the drones from a distance.

Nobody knows how to design drone fleet management routines. It is entirely a matter of trial and error, or test-and-see. That's why insurers tend to work with small specialist companies that are amassing this test experience from multiple projects. The principles of experimentation or testing are entering every sphere of business. We believe that the Flow technique of small units of work tested at the point of acceptance is a good model for testing, and hence innovation, everywhere.


What you can also see from this short description is that new roles are inevitable. There was no such thing as a drone fleet manager 10 years ago; maybe three years ago. But it will become more commonplace. There are no "testers" or 'test writers" in many areas of business. There are QA people who have a role in ensuring that technologies meet a particular performance threshold. But we believe testing will become a specific function across the business some time very soon.

All Posts

Almost done…

We just sent you an email. Please click the link in the email to confirm your subscription!

OKSubscriptions powered by Strikingly