System Level Product Development Waterfall or Agile? Do Both!

Below are three verbatim quotes from engineering managers attending a presentation we gave for PMI.

  • “Agile and Waterfall development processes are like oil and water. They don’t mix.”

  • “Applying Agile is all or nothing”

  • “You must be able to ship every two weeks”

These are real beliefs held by real managers!  We hear similar statements frequently from hardware teams that are trying to increase speed and predictability in their product development projects. The reality is that hybrid processes that mix the discipline and long-term planning of Waterfall with the adaptability and speed of Agile are not only possible – they are best-in-class.  Also, this hybrid is the best application of Agile in hardware development, where milestones around the release of tooling designs and readiness to scale mass production are essential.  If a milestone or review is not held, there is elevated risk that enormous delays or significant financial exposure could occur.  It would be imprudent to not perform these check-ins.

Myths Hinder Adoption

The myths about Agile generally come from two groups.  The first, and high in number, are the Agilists who have almost a religious view of ‘Capital A’ Agile.  The second group is comprised of more tenured hardware engineers and their managers.  They are both wrong, but for different reasons. They’re correct that a rigid application of Agile software techniques within a hardware development project breaks down quickly. However, when modified for the unique characteristics of hardware design and manufacturing, key elements of Agile allow teams to gain the speed, predictability and product definition advantages that their software counterparts enjoy.

Figure 1: Agile Manifesto – Not Just for Software

Figure 1: Agile Manifesto – Not Just for Software

The Agile Manifesto – Not Just for Software

Our experience with clients, as well as our primary research, explode the myth that Agile is all or nothing and that it is inapplicable to tangible products. The Agile manifesto itself refutes the notion that Phase-Gate/Milestone and Agile are mutually exclusive. Of the twelve Agile principles contained in the Manifesto, 75 percent are not specific to software. For example, such concepts as “Business people and developers work together daily”; “Projects require motivated individuals, support & trust” or “Face-to-face conversation is most efficient” are not software specific. There’s no reason that these principles cannot operate in any product development environment. Segments such as Medical devices, with hardware and software elements, within a highly regulated environment, benefit from nesting Agile within a Waterfall process. 

There are significant differences between tangible products and software, but nothing prevents developers from importing select principles from Agile to other environments. Research we conducted in 2014 showed that the use of sprints, having more iterations, and developing a local build capacity were used effectively by tangible product developers. Our experience with clients shows that selective use of tools from the Agile toolbox is not only possible but that it also reduces team burnout, brings teams closer to customers, and improves the adaptability of product teams to shifts in markets or technologies. It’s not all or nothing.

Embedding Agile Within Waterfall

Applying Agile to hardware products does not have to be an “all or nothing” process. If teams have a phase gate or milestone-based process, they can achieve the best results by applying modified agile methods within their current process. The key to gaining this competitive advantage is to divide long product development timelines (12-18 months) into sprints, a tool frequently used in Agile software. Then, nest these sprints within the Phase-Gate/Milestone framework. Using sprints within this larger framework accelerates decision-making. Sprints also divide a long journey into smaller steps, with concrete deliverables, within a fixed duration of from two to four weeks. This division of the product development timeline into smaller units increases urgency and keeps the team on task. The results are less waste and faster product development timelines.

Figure 2: Nesting Sprints in Waterfall

Figure 2: Nesting Sprints in Waterfall

But developing tangible products also requires the discipline and planning that Phase-Gate/Milestone processes afford. These longer-term cycles of planning provide the financial controls that tangible products require. Unlike many software products, developing tangible products tends to involve numerous functions, partners and suppliers. Unlike software, they often involve tooling, material buys and inventory control. Traditional, gated processes also prevent budgets from ballooning out of control. For many products, regulatory or qualification cycles are also essential facets of the product plan. For these reasons, tangible products require an end-to-end product plan, with a comprehensive schedule to ensure that launch requirements are satisfied.

Embedding Agile - Tips and Tricks

Use Release Planning to manage dependencies:  In working on system level products that involve operations, purchasing, safety & regulatory in addition to many other functions, there are often external dependencies and decisions (such as approving a new supplier).  Some of these activities don’t normally surface in the sprint level planning.  At the beginning of each phase (or phases), use a sprint planning session to determine which sprint resolves external dependencies.

Eliminate status meetings:  With an iterative approach, with progress tracking inside sprints, and demos at the end of each sprint, management is better informed about the team’s progress.  There is no need for weekly status meetings.

Modify The Definition of ‘Done’:  Some hardware deliverables span more than one sprint; so how do you deal with these deliverables when you plan sprints and write your end of sprint ‘Acceptance Criteria’?  The solution is to require senior technical people to break down long lead time items (such as tooling Printed Circuit Boards) into smaller measurable pieces. For example, have your PCB partner describe several of their internal milestones and reveal them to you.

Adopt Agile Principles Incrementally

At its heart, adopting Agile methodologies requires organizational change management.  It does not happen over-night, and in large organizations, not even quickly. A proven approach is to pilot a project, using team members that have a track record for embracing improvement. Organizations learn quickly from a pilot and then improve incrementally. Agile gurus and guidebooks are sometimes too rigid, imposing an orthodoxy that doesn’t make sense for your culture or your products. Start small, learn the basics, and then improve by modifying the tools to fit your environment.

High Performance Teams – the Secret Sauce

We’ve discovered that much of Agile’s success depends upon high performance teamwork. That means:

1) Upper management has empowered teams to plan and prioritize their work;

2) Team members have both domain expertise and collaboration expertise;

3) Middle management changes their focus from micro-managing to removing roadblocks;

4) Everyone puts product allegiance over functional allegiance and,

5) Teams have dedicated Product Owners and Scrum Masters.

High performance teamwork requires a new way of working, but the most successful companies master this challenge and reap the rewards.

Missed Opportunity

Many teams are losing an opportunity to accelerate product development because of unexamined assumptions about Agile. The myths create one of the tallest barriers to adoption. A lack of prescriptive best practices for adapting Agile principles to tangible products is another hurdle. But changing minds and adapting what works is possible. In fact, it is necessary for staying ahead of the curve of product development speed.  Companies that have mastered teamwork are more than half the way there.

Another key success factor is to define the sprints that are nested within your phase-gate process very carefully. Use short intervals, and define the goals, deliverables and metrics for each sprint employing the Tips and Tricks covered above.

Finally, give yourself adequate time to pilot and adopt Agile principles to your environment. Corporate culture tends to play a larger role in companies that develop tangible products and change takes time. The culture must adapt to the fact that, when you bring Agile tools to the work, functional managers must cede some control. Since teams that develop tangible products tend to be cross-functional, there are more functional managers, that is, more cooks in the kitchen. This can be a challenge when your team is trying to pilot sprints, daily meetings, and other Agile tools that help to accelerate your teams. Again, the key is to start small, build incrementally, and remain persistent with your Agile adoption.