Making progress — in society, a team, or life — isn’t straightforward most of the time. Knowing where you want to go is generally the first step, but the destination can be very broad. And even if there’s a specific goal, the path to get there may be very indirect.
As strategy transitions into execution, it’s important to understand how these attributes affect progress. If an effort is managed or guided the wrong way, it may be doomed to failure no matter how difficult it is.
Project management as a discipline works great, but not with everything. Managing large, more uncertain endeavors in particular has been problematic recently. The wide-scale ongoing effort to fight COVID-19 and return the world to normalcy brings this challenge front and center.
Why are certain efforts harder than others and how do we navigate them? How were we able to accomplish such large scale collaborative efforts such as the Apollo program or Manhattan Project, but can’t do the same thing for curing cancer?
If we want to build, we need to understand the answers to these questions. The following is a framework for classifying efforts by the certainty of both their objectives and the paths to achieve them. Knowing which “mode” an effort is in is critical to understanding and managing its progress.
How do we classify efforts into modes? The best paradigm I’ve come across is the how/what quadrants.
In his 1994 book “All Change!”, Eddie Obeng described 4 different types of projects along with the difficulties and peculiarities of each: quests, movies, painting by numbers, and fog. It turns out putting a project on both the know how and the know what scales tells you a lot about how it should be managed.
Venkatesh Rao explored the concept much further in his essay on the “Grand Unified Theory of Striving”, pulling in other ideas like convergent thinking, critical paths, and lean methodology. Venkat’s visualization of the critical paths and point frontiers of each quadrant is a particularly insightful way to think about the concept.
Defining the dimensions and modes
Here’s how I’d describe the axes of the 2×2:
“Why”-axis — Know what vs. don’t know what. Do you know what the goal is? How specific is the desired outcome? Not knowing the goal (or having a very broad idea) is in the realm of divergent thinking: there are many potential solutions and progress can be non-linear. It’s the exploration phase of the explore vs. exploit tradeoff, searching for goals or areas of value.
Knowing what and why is in the realm of convergent thinking: there is a single “correct” solution or destination. It squarely aligns with Peter Thiel’s deterministic approach of viewing the future: “There is just one thing—the best thing—that you should do.”
Horizontal-axis — Know how vs. don’t know how. Is it known how to accomplish the goal? Are the bottlenecks or resource-sensitive parts generally understood? When you know exactly how to accomplish something, there is a clear critical path1 (the red lines in the diagram below). Other paths of effort may still be required, but they are oblique with more slack, running parallel to the critical path.
Knowing how allows you to operate lean because you can—in theory—use the least amount of resources necessary to get the job done. In the fat mode of operation, you don’t really know how to reach your goal. You can’t be efficient because you don’t know how to be, and there will be a lot of slack in the system. The path is determined opportunistically as you go, with critical paths only in smaller subsections.
“An old story tells of a visitor who encounters three stonemasons working on a medieval cathedral and asks each what he is doing. ‘I am cutting this stone to shape,’ says the first, describing his basic actions. ‘I am building a great cathedral,’ says the second, describing his intermediate goal. ‘And I am working for the glory of God,’ says the third, describing his high-level objective. The construction of architectural masterpieces required that high objectives be pursued through lesser, but nonetheless fulfilling, goals and actions.”
John Kay, Obliquity
All efforts, from daily personal projects to global collaborative endeavors, fit in a webbed hierarchy of abstraction.
Understanding the full hierarchy of an effort is critical to accomplishing it, along with its higher-level objectives in the long-term. Not understanding it can result in bad planning, mismanagement, and failed expectations.
Ladder of Abstraction
“…the most powerful way to gain insight into a system is by moving between levels of abstraction.” — Bret Victor
The ladder represents a top-level concept or domain, with each rung a subset of the one above it. The rungs move from abstract at the top, to concrete at the bottom. The lower down, the more detailed and specific. The higher up, the broader and more abstract the concept.
The model is very simple, and can be applied to almost any discipline with a hierarchy of nested groups. This includes applying it to efforts.
First of all — what do I mean by effort?
An effort is the active search for the best outcome of an objective. It encompasses both the objective and the pursuit of that objective — both of which are not fixed and can evolve over time. The objective always has some boundaries, but otherwise can be very broad (“solving climate change”) or very narrow (“double next-month’s sales volume”).
All efforts are multi-scale and nested.1 This means we can put them on a ladder of abstraction, each rung with an objective or method that’s a prerequisite of the one above it. Lower-level goals are nested in higher-level purposes. Good project managers do this intuitively when breaking an objective down into tasks and sub-goals, mapping their dependencies.
Because efforts can have many dependencies and relationships aren’t just one-to-one, they exist in more of a webbed hierarchy of abstraction than a ladder. A simple one-dimensional ladder of abstraction is just a slice of the larger hierarchy.
Here’s an example of a hierarchy of abstraction for the efforts relating to Covid-19:
What’s the best way to determine the hierarchy of abstraction for an effort?
A simple way to move up and down the ladder is the Why/How Chain. To move up, ask “Why?”; to move down, ask “How?”. Many know this technique from the Toyota Production System’s method of asking 5 Why’s to find the root cause of an issue.
You can start by finishing the phrase: In what ways might we ___? This method can work on almost everything, from large-scale efforts to small-scale jobs-to-be-done:
⬆️ Why? To make my home look good.
⬆️ Why? To hang a picture.
❇️ In what ways might we drill a hole?
⬇️ How? Use a drill.
In the Covid-19 example, you could start at whatever level is most relevant to you.
❇️ In what ways might we provide better medical care for COVID patients?
⬆️ Why? To stop people from dying and reopen the economy.
⬇️ How? Protect medical workers from getting sick.
⬇️ How? Source and distribute PPE.
⬇️ How? Contact regional manufacturers.
There will always be multiple “how”s, which is the essence of breaking a goal down into sub-goals. There can be multiple “why”s as well, especially the further you go down the ladder. But high up in the hierarchy the whys and hows become more and more vague. This means you have to approach them in a completely different way.
Abstraction = Obliquity
Knowing where an effort fits on the hierarchy is the first step. Now we need to understand how the different levels of scale need to be treated.
This is where John Kay’s concept of obliquity, from his book of the same title, comes in.
To solve a problem obliquely is to solve it through experiment and adaptation. In general, the bigger the scope and complexity of an objective, the more indirect the path is to achieve it.
The ladder of abstraction is a proxy for obliquity. The higher on the ladder, the more adaptive the problem should be solved. John Kay: “High-level objectives — live a fulfilling life, create a successful business, produce a distinguished work of art, glorify God — are almost always too imprecise for us to have any clear idea how to achieve them.” In the process of making progress on these objectives, we don’t only learn how improve, but “about the nature of the objectives themselves.” You’re wayfinding, rather than following a prescribed path.
The lower on the ladder, the more direct. “Directness is only appropriate when the environment is stable, objectives are one-dimensional and transparent and it is possible to determine when and whether goals have been achieved.”
The following table compares the different aspects of both ends of abstraction.
Clear and simple
Loosely defined and multidimensional
Most outcomes are intended
Outcomes arise through complex processes with no simple cause and effect
Interactions with others
Limited and predictable
Dependent on many variables, including interpretation of them
Range of available options is fixed and known
Only a subset of options are available from successive limited comparison
Can be described probabilistically
Uncertain: Range of what might happen is not known
Insists on consistency: always treating the same problem the same way
Consistency is minor and possibly dangerous — rare that same problem is encountered twice
Conscious maximization of objectives
Adapt continuously to changing circumstances
Consistency is vital when you’re low on the ladder, not so much higher up. “The oblique decision maker, the fox,” John Kay remarks, “is not hung up on consistency and frequently holds contradictory ideas simultaneously.”
But the real power of solving an oblique problem lays in adaptation: “If the environment is uncertain, imperfectly understood and constantly changing, the product of a process of adaptation and evolution may be better adapted to that environment than the product of conscious design. It generally will be.” There is no map, so instead you have to wayfind and look for clues in front of you, making your way with the tools you have at hand.
Keep in mind again that this is a scale — it’s rare that an effort would completely check all the above boxes for either Direct or Oblique. The point is that efforts always fall somewhere on the scale and that this determines the best methods to pursue them.
Consequences of Misplaced Directness
I’ll try to keep this section short, as whole books have been written on the consequences of misplaced directness. See Nassim Taleb’s Incerto for example.
Attempting to approach a large, complex effort too directly almost always leads to failure — or at the very least a failure to meet initial expectations. Directness is only appropriate when the objective is one-dimensional and the path to achieve it is known.
The intention was to create a new Brazilian capital from scratch that was truly unique and modern, paying special attention to cars and traffic flow. (This was the same time period the U.S. began building out the intercontinental highway system.)
As time went on, unforeseen circumstances in the messiness of the real world intervened. Overpopulation drove traffic congestion, slums, and general inequality. The focus on form over function from the top-down design caused alienation and poor quality of life. This is exactly why any such an effort can’t be planned with precision. Not only are the details of the true goal not understood, but the methods to achieve it involve unpredictable complexity. They’re in the world of “extremistan” as Taleb would say.
“The structures in this artificial capital are impressive,” read an FT article on one of the architects, “yet few want to walk its barren streets. Politicians leave as soon as possible to return to grittier, but livelier, Brazilian cities.”
The Brasilia master plan was partly based on architect Le Corbusier’s misplaced utopian vision of creating the ideal city. Corbusier’s work also included the Indian city of Chandigarh with similar consequences. This was, in the words of John Kay, “the hope that rational design by an omniscient planner could supersede practical knowledge derived from a process of adaptation and discovery.”
Many overfunded startups suffer from the same fate. When you have access to seemingly unlimited resources, it’s easy to be fooled into thinking you can build your exact vision into reality. But these visions generally exist in a complex world of human culture, desires, and economic feedback loops.
Quibi — the $1.8 billion funded short-form media startup — is case in point. Ultimate success or failure remains to be seen but results have yet to come close to expectations. They had the resources to build a particular product and business model into reality — a reality where most customers don’t seem to want what they’re selling. A destination was chosen on a map that couldn’t be seen.
Magic Leap has an amazing vision of seamless AR glasses to enable a digital layer on top of the real world. The actual objective in this case might actually be the right destination. But the complexity of the problem means the approach still can’t be direct. Currently their product has seen minimal success as they struggle to find a sustainable business model.
WeWork seemed to start at the right level of abstraction, then an ambitious “visionary” founder is given unlimited funds, and a direct approach is applied to a still-oblique problem.
Summary: The Right Strategy for the Right Level
All efforts, and the efforts within them, can be placed on a ladder of abstraction. The higher up you go, the less concrete the objectives and less straightforward the methods to achieve them.
Where the effort falls on the scale is critical to the strategies for making progress on them. Direct, methodical approaches are only appropriate at lower, smaller-scale levels. This is where it’s good to plan details, use processes, and keep things consistent.
You can still have a grand, abstract vision. You just need to wayfind to get there: working from the bottom up, adapting and evolving the path while shaping and refining the details of the goal. Keep things flexible, adaptive, and opportunistic at the top.
Organizations: Given the definition of an effort above, what about coordinated groups of efforts or goals? ↩︎
In the book The Origin of Wealth, Eric Beinhocker describes organizations as “goal directed, boundary-maintaining, and socially constructed systems of human activity. . . . There is a boundary distinguishing the inside world from the outside world, and the goals of the organization drive activities that lower entropy inside the organizational system.” This is the best description I’ve come across given its abstract nature. But I’d like to propose simpler, yet still compatible definition.
An organization is a group of people pursuing one or more ongoing efforts, generally with the same high-level objective. This means an organization can be anything from seven-person hunting parties, to fleets of exploratory vessels, to philanthropies, to a multinational conglomerate.
In April, Marc Andreessen put out the call to build. It was in response to our failure to control and mitigate the effects of Covid-19 — institutions on every level were unprepared for the pandemic, and have continued to show their inability to quickly find and scale solutions.
But more than anything it was in response to our failure to build in general. We chose not to build, he claims. “You see it throughout Western life, and specifically throughout American life.” The problem isn’t a lack of resources or technical ability — it’s with supply and demand of desire. Demand is limited by our ambition and will to build. Supply is limited by the people and organizations holding it back.
Andreessen is generally an optimist, which is why I see his essay as positive in overall tone. But it was also somewhat of a mea culpa. Andreessen has for years been on the other side of Peter Thiel’s view of modern technical stagnation.
Thiel’s view may be too pessimistic, but there’s a kernel of truth to it. If you’re familiar with the history of tech and innovation, something feels different. The late-1800s to mid-1900s had explosions of innovation in fields from medicine to consumer products, transportation, energy, communication, computing, food, and more.1
This is the introduction to a series of ongoing essays centered around the question:
What frameworks can help us build more, better?
And further attempting to investigate the answers to the following:
What are the best ways to approach solving big, complex problems?
Why are certain efforts so much harder to achieve than others?
How are these efforts best managed at every level?
How do we build things faster? (Without sacrificing quality or safety.)
What is holding us back from building more?
How do we overcome these barriers?
Many of these lessons apply not just to “building” in the physical sense, but for solving problems, scientific discoveries, improving systems, and making progress overall. Building in a way is symbolic. It represents making big, necessary changes to move humanity and our planet forward. This can be building something physical or digital, pushing the boundaries of fundamental research, or trying new uncertain ways to solve problems.
It doesn’t even have to be anything new or innovative per se. Andreessen gives many examples of expanding existing tech: housing, infrastructure, education, manufacturing. Even preservation and restoration — in many ways opposites of building — can still apply. In the early 1900s, President Teddy Roosevelt established over 230 million acres of public lands and parks. This added an incalculable amount of value to future generations. I would love to see E.O. Wilson’s Half-Earth Project executed at scale. This is in the spirit of building: making progress and pushing humanity toward a better future.
Here’s a preview of some of the specific topics I want to explore in the series: Ladders of Abstraction (why/how chains), Oblique vs. direct approaches, Modes of effort (why/how quadrants), traversing fitness landscapes, the explore vs. exploit tradeoff, the role of trust in building things fast, forcing functions, and the specific methods we used to accomplish large-scale collaborative efforts such as the Apollo program, the Manhattan Project, etc.
Wayfinding Through the Web of Efforts[8 minutes] — Putting goals on a ladder or hierarchy of abstraction. Defining efforts and their multi-scale nature. Determining the hierarchy of efforts using a why/how chain. The difference between making progress directly and obliquely, and the consequences of misplaced directness.
Managing Modes of Effort[10 minutes] — A framework for understanding how managing progress differs across scales of effort. Classifying efforts into four modes on the how/what quadrants. Defining the modes and how they fit on the hierarchy of abstraction. A Covid-19 case study. How to manage an effort based on its mode.
What was different about this era? The following is a good rundown from Vaclav Smil’s book “Creating the Twentieth Century” on the remarkable attributes of the pre-WWI technical era:
The impact of the late 19th and early 20th century advances was almost instantaneous, as their commercial adoption and widespread diffusion were very rapid. A great deal of useful scientific input that could be used to open some remarkable innovation gates was accumulating during the first half of the 19th century. But it was only after the mid-1860s when so many input parameters began to come together that a flow of new instructions surged through Western society.
The extraordinary concentration of a large number of scientific and technical advances.
The rate with which all kinds of innovations were promptly improved after their introduction—made more efficient, more convenient to use, less expensive, and hence available on truly mass scales.
The imagination and boldness of new proposals. So many of its inventors were eager to bring to life practical applications of devices and processes that seemed utterly impractical, even impossible, to so many of their contemporaries.
The epoch-making nature of these technical advances. Most of them are still with us not just as inconsequential survivors or marginal accoutrements from a bygone age but as the very foundation of modern civilizations. ↩︎