Agile doesn’t jibe with big, so the big question is: how do you scale Agile?
The good news is, Agile is common sense, not rocket science.
You can be Agile in a big way when you move beyond the basics and pay attention to three things:
- Organization as a whole,
- Organization using containers,
- Container integration.
Some Background (or skip this part)
Once upon a time, people developed applications using a “waterfall” process with a few simple phases: requirements, design, code, test, and voilà—release.
This “software development life cycle” (SDLC) made sense and people were thrilled and thorough (I was one of them), taking great care in “gathering” requirements and mapping them from phase to phase to make sure the final solution would meet those carefully-gathered requirements.
Then, in 2001, a group of independent-minded software professionals got together to “talk, ski, relax, and try to find common ground.” The gathering was a success: it resulted in the Manifesto for Agile Software Development, or as you call it, the Agile Manifesto.
The Agile Manifesto called attention to a carefully selected set of values and principles (four values and twelve principles, to be exact) and proclaimed a better way of working, and so Agile was born.
Fast forward to today, Agile is everywhere from startups to corporate America, in part thanks to the most popular incarnation of the Agile value system: the Scrum framework.
What Is Agile About?
At its core, Agile is adaptive rather than predictive, and people-oriented rather than process-oriented.
Another tenet of Agile: breaking big, overwhelming things into small, manageable chunks.
Break big, overwhelming things into small, manageable chunks.
“Breaking big into small” works: it reduces complexity and increases agility across areas like people (go from large groups of people to small teams), projects (go from lengthy requirements documents to small stories), process (go from long development phases to short sprints), and, with cautious optimism, architecture (go from monolithic applications to microservices).
And so, thanks to Agile, companies changed.
Before Agile, companies spent months painstakingly developing elaborate solutions that users didn’t necessarily want. Bummer.
Now, companies deliver delightful functionality continuously, thanks to autonomous teams that change course along the way based on customer feedback. Hurrah!
Is that it?
Not quite. If Agile were that simple, there wouldn’t be much to write home about. Unfortunately, as the saying goes, if it seems too good to be true, it probably is.
Agile Gone Awry
Back in 2001, things certainly got off to a strong start with a crisp manifesto and a perfect word choice: “agile.”
But then, 10–15 years later, big companies jumped on the Agile bandwagon, bringing in process and tooling specialists who promoted their process or tool as the way to be “Agile at scale”—and things got dicey.
No doubt, there’s value in processes and tools. It’s just that basic mechanics and fancy tools will only take you so far.
Not only that, at scale, processes and tools are insidious: they become treated as ends in themselves, creating complexity that slows down—or even undermines—your Agile transformation, instead of accelerating it.
Before you know it, processes and tools dominate as one-size-fits-all solutions, leading to oversimplification in some areas and overcomplication in others.
In the end, processes and tools force people to accept undesirable compromises and distract them from the real goal: being organized to act together and get the right work done, the right way.
Scrum
Take Scrum for example.
For big organizations, it’s a no-brainer to start with Scrum, because Scrum has prevailed as the de facto standard of Agile frameworks.
Basically, Scrum is a safe way to go.
Scrum is not only a safe way to go, it’s also straightforward: the Scrum Guide defines Scrum with three roles, five events, and three artifacts.
Scrum is good because it’s a simple way to break down monolithic organizations and overwhelming projects into manageable chunks. In a nutshell: (1) you divide large groups of people into small teams (the Scrum Teams), and (2) you ask each team to work with a business-minded person (the Product Owner) to turn projects into pieces of functionality (the Product Backlog items) that the team can implement within short time boxes (the Sprints).
Scrum is good but Scrum leaves a lot to be desired.
For one thing, if you apply Scrum by the book with dozens or hundreds of teams, you’ll certainly end up with small teams doing work in small chunks and short sprints. However, the visible and hidden dependencies between teams, chunks of work, and sprints (not to mention dependencies on non-Scrum teams and non-Scrum processes) will leave you wondering how much agility you really gained.
The answer: not much.
Sure, you can double down on Scrum and go with a Scrum variant meant for scale, like Scrum@Scale, Nexus, or Large-Scale Scrum. But chances are, “Scrum on steroids” will still fall short of addressing your reality and meeting your expectations.
There’s only so much that any one process can do, and the same goes for tools like Jira, Rally, and the like.
So, your choice:
A. Use processes and tools to roll out “Agile” en masse across dozens or hundreds of teams and settle for a sub-optimal organization that’s marginally better than what you started with.
B. Stand by the first statement of the Agile Manifesto: “Value individuals and interactions over processes and tools.”
Keep reading if you chose B.
—
Container Structure
If traditional processes and tools aren’t the answer, then back to the big question: How do you scale Agile?
The answer: create your own multi-container structure.
Unlike traditional processes and tools, which impose methods on people and reduce interactions between them, a multi-container structure empowers people to organize themselves at different levels (from the big picture to the nitty-gritty details) and across multiple dimensions (people, products, projects, and practices).
Now please bear with me, there’s quite a bit to unpack.
Containers
The container construct is nothing new: containers come in all shapes and sizes.
Back to Scrum for a second, Scrum thrives on small containers: small teams (for people), small backlog items (for work), and short sprints (for time).
In technology, services and microservices are containers of sorts, and so are, well, application containers.
Containers are everywhere and for good reasons: containers shield you from external interference and increase your focus.
Containers are an effective means to break big, overwhelming things into small, manageable chunks—the Agile tenet mentioned earlier.
So, can containers transform large monolithic organizations into Agile federations?
Containers certainly can—when you “containerize” all the way, step by step.
Simply follow these 10 steps.
Step 1: Start with One Big Container
To begin, start with a “big container.”
With this big container, you want to carve out and encapsulate a piece of your business so you can handle it in an Agile way, as a smaller company within the company.
Whatever you call your big container, think of it as a company in its own right—a startup or “mini-company,” if you will.
Your big container should be as big as possible, but not so big that it becomes impractical (“too big to succeed”).
Your big container should include these four basic elements:
- People,
- Products (at least one),
- Projects (or “epics”), and
- Practices.
It’s good to start this way so you can improve the whole (optimization), not just the parts (sub-optimization).
Step 2: Nest Containers
Next, think of containers as matryoshka dolls and form “nested container structures.”
Be deliberate about the number of levels in your structure, and calibrate the size of the containers until you achieve something that works for you. For example: 3–9 people make a team, 3–9 teams make an area, 3–9 areas make a group, and 3–9 groups make up the mini-company.
The military provides a good example of this type of nested structure: 2 fireteams make a squad, 3 squads make a platoon, 3 platoons make a company, and so on.
Step 3: Create a Taxonomy of Containers
Use the same type of nested container structure to organize the other dimensions mentioned in Step 1: Products, Projects, and Practices.
The Project category is for work: programs (or “initiatives”), organized into actual projects (“epics”) that are implemented in small chunks (“stories”).
The Practice category is for how you get work done: management practices and technical practices, each with their own areas, tools, and techniques.
The Product category is for technology assets (applications, platforms, services) and actual products (offerings your customers are or would be willing to pay for).
Be deliberate with how you name your containers. Take Spotify, for example: with labels like “tribe,” “chapter,” and “guild,” Spotify created a vernacular that reinforced its unique culture.
As ordinary or creative as your labels may be, the important thing is to define, group, and nest containers into categories that work for you.
In technical terms, you want to create a taxonomy of containers—a taxonomy you can call your own.
Just remember, the goal is to develop a container taxonomy that defines the most elegant and powerful multi-dimensional organizational structure possible.
Easy peasy.
Step 4: Align Containers
The more containers you use, the more complexity you add—and complexity is the enemy.
One way to reduce complexity is to align containers across dimensions, because when containers align, they essentially become one.
Another way to reduce complexity is to keep the number of levels to a minimum.
For example, align three levels for People (teams, groups of teams, mini-company) with three levels for Projects (stories, epics, initiatives) and three levels for Products (product areas, products, product line).
Put another way, at each level, the span of the containers should be relatively the same.
Step 5: Use Extra Containers
How nice would it be to organize every thing into neat stacks of nested containers?
Every element of the whole would simply belong to one container inside another: people inside teams and groups of teams; work inside epics and initiatives; components inside applications and products.
In reality, as you develop your container structure and its taxonomy, you’ll want the flexibility to layer additional containers to further optimize the whole.
In the People category, you can use containers to manage important variables, starting with location: neighborhoods within buildings, buildings within cities, cities within time zones.
From a people management perspective, area of expertise is another important variable: when people report to a manager based on their area of expertise, they can switch teams (their primary container) while staying with the same manager.
In the Project category: releases across projects.
In the Practice category: metrics.
In the Product category: customer experiences across products, applications, and communication channels.
Basically, you can use extra containers to pay extra attention to different aspects of the overall organization.
Step 6: Use Makers and Enablers
Distinguish between two types of container: “makers” and “enablers.”
Agile values independent teams of makers who build products.
Meanwhile, enablers support makers.
Enablers can provide tools or specialize in functions like release automation or information security, among other things. They make makers more effective and increase control.
Agile overlooks enablers because they don’t deliver value directly. What’s more, in practice, enablers can underserve and undermine makers, causing more harm than good.
You also want to encourage makers to take care of themselves—acquiring the skills, capabilities, or team members they need to operate independently, unencumbered.
That said, enablers make sense as your organization scales, and you’d be remiss not to give them the attention and support they deserve.
Here too, the military provides good examples:
While a squad (“maker”) benefits from having combat lifesaver training and equipment within the squad, it’s also better off with the support of a medic attachment (“enabler”).
Support increases at the platoon level for specialty functions (combat medic, radio telephone operator, etc.), and as you move up the levels, it expands to fulfill staff functions (personnel, intelligence, training, supply, etc.).
Combat units and support units, makers and enablers—you get the gist.
Step 7: Integrate Containers
By now, you have a lot of containers, and as a result, the complexity shifts from the containers themselves to the connections between them.
Now the challenge isn’t so much with what’s inside the containers, but with the dependencies and interactions between the containers.
When you integrate systems, you meticulously design and document interfaces (APIs). This way, teams can work independently on their own systems, knowing how everything will come together in the end.
The same goes for containers: you want well-defined interfaces between containers to ensure they form a cohesive whole, with no gray areas or gaps.
As you define interfaces between containers, it’s a good time to confirm boundaries. Containers should be as independent as possible, and interfaces as simple as possible. In technical terms, you want containers to be well-encapsulated and loosely coupled.
Take “makers” and “enablers,” for example. Makers and enablers should work out how they’ll interact and build a solid bond: how the enabler will support the maker and, in turn, how the maker will inform the enabler.
As you define interfaces, just stay true to this core principle of the Agile Manifesto: value collaboration over contract negotiation.
Step 8: Operate at Different Levels
“Vision without execution is a daydream. Execution without vision is a nightmare.”
In large organizations, the situation on the ground is always more complicated than it seems, and it’s no surprise to see gaps and waste between vision and execution.
One way to bridge gaps and reduce waste is to deliberately operate at different levels: from entire organizations to individuals, from strategy to implementation details, from architecture to code.
With a container hierarchy, you can explicitly define levels to break down and compartmentalize work.
Once the levels are clear, it becomes a matter of doing “just enough” work at the right level and at the right time—with higher levels providing context and lower levels providing validation.
Step 9: Federate
Agile encourages teams to self-organize, but there’s obviously a limit to how far you want to take this principle. For example, for teams working on the same application, it makes sense to share common practices and tools, starting with a common source code repository.
Along the same lines, at higher levels of organization, self-organization brings up the dilemma of centralized versus decentralized governance.
Monolithic organizations favor centralized governance and tend to standardize on single solutions. With this approach, you’re essentially back to square one: one-size-fits-all processes and tools.
In contrast, Agile-style organizations strive to keep central governance to a minimum, empowering teams to work as they see fit.
At scale, with hundreds of teams, no style is perfect. The monolithic style can lead to sclerosis, while the Agile style can turn into chaos. And a fuzzy mix of both styles is no better.
Enter containers and the “multi-container” style of organization.
You started with one big container (Step 1). At this point, to transform the entire company, you’ll want to use not just one but several big containers.
Beyond using big containers to carve out discrete mini-companies, the goal is to create a federation of big containers, each managing its own internal affairs.
Establish just enough overall structure to operate as a unified whole, then step back and empower people to organize themselves as they choose.
And don’t stop there: apply the same approach at every level, treating each container as a federation of smaller containers within it, all the way down to teams.
Your goal is to create your own kind of federated, multi-container structure that enables you to deliberately give and organize control at different levels of the organization.
Again, terms like “big container” aren’t what’s important—use your own labels. What matters is the approach of using containers to break down entities that are overwhelmingly large into concrete parts that are as autonomous as possible.
Step 10: Be Bold
It sounds good to use containers to break monolithic organizations into autonomous parts, just as it sounds good to be Agile.
It sounds good but it’s hard to do.
It’s hard to do because it’s difficult to decide where container boundaries should lie. And once you do, reorganizing to make those containers a reality is just as hard.
There’s also this natural desire to break down silos and develop universal solutions that should in theory lead to cost savings and a seamless customer experience.
For example, if you were Google when it bought Waze in 2013, wouldn’t you have been tempted to go right ahead and merge Waze and Google Maps?
But you can’t have it all—and Google kept Waze and Google Maps separate.
Agility comes at a price: you have to be willing to carve out concrete entities, and you have to slice around them to give them some autonomy.
To achieve this, referring back to “makers” and “enablers” (Step 6), you can favor either makers or enablers.
If you focus on makers, your ultimate goal is to divide the organization around business lines and operate as a portfolio of mini-companies that are virtually free and independent from each other.
By the way, this approach—operating as a portfolio of mini-companies—doesn’t mean you have to abandon the goal of creating seamless customer experiences. For example, back when it launched in 2007, Mint.com demonstrated that it’s possible to create an independent solution that consolidates all your bank accounts in one place, seamlessly, no matter how many banks you use.
On the other hand, if you focus on enablers, your ultimate goal is to develop solutions that are so good that other companies, not just yours, would want to use them.
Amazon provides a good example of how you can operate in both maker and enabler modes at the same time.
Maker mode: In 2009, Amazon acquired Zappos under an agreement that the relationship between the two companies would be “governed by a document that formally recognizes the uniqueness of Zappos’s culture and Amazon’s duty to protect it.” To this day, Zappos operates as an independent part of Amazon.
Enabler mode: Since 2006, Amazon has been developing Amazon Web Services (AWS), “the world’s most comprehensive and broadly adopted cloud platform, offering over 165 fully featured services from data centers globally.” Today, AWS supports over one million active customers and has grown into a highly successful company in its own right.
Whatever modes you choose, have a clear strategy and communicate it so everybody is on the same page. Incremental implementation is good, not incremental thinking.
Incremental implementation is good, not incremental thinking.
The good news is that it doesn’t have to happen overnight (the AWS idea took years to develop), and it’s not a once-and-done thing. Containers take time to emerge, and container boundaries should continue to evolve.
Just be bold.
In Summary
- Start with one big container
- Nest containers
- Create a taxonomy of containers
- Align containers
- Use extra containers
- Use “makers” and “enablers”
- Integrate containers
- Operate at different levels
- Federate
- Be bold
Final Thoughts
The term “container” is good because it makes it clear you want concrete, self-contained entities that can be managed in an Agile way. “Container” also compels you to define boundaries, instead of being fuzzy on the edges.
And a “multi-container” structure is good because it’s a simple yet powerful way to see, simplify, and optimize your multi-level, multi-dimensional organization as a cohesive set of containers.
See, simplify, and optimize your organization as a cohesive set of containers.
But obviously, all of this is easier said than done.
Let’s be honest, if your organization is big and complex but your business is doing well, you may find it reasonable to keep things more or less as they are and settle for one-size-fits-all solutions.
That said, if you want agility at scale and you mean it, the multi-container method is a solution worth pursuing: use containers, be bold and deliberate, and you’ll pave your own path—big time.
