Tech Stack Trees

Every product is built on and enabled by one or more technologies.

Understanding where a product fits on its higher-level tech stack is an important part of any long-term strategy or investment thesis.

The following is an exploration of tech stacks: what they are, how to model them, and what roles their components play. I also discuss what insights can be gained, such as potential opportunities to expand the business.

Stack Trees

Typically, a tech stack shows what infrastructure a technology is directly built on or requires. A SaaS startup for example could have a front- and back-end software stack with a cloud provider like AWS under it. The tech in focus is on top of the stack, with the supporting layers below it.

A tech stack tree is a higher-level version, branching both above and below the “layer” in focus. It shows both what the technology directly relies on and what relies on it. Stacks are fractal in nature, just like trees. An innovation spawns many others that use it, which further enable others, and so on.

A stack tree shows the relevant “slice” of the full dependency graph, going only a few nodes out. It looks something like this:

How to model a stack tree

Step 1: Determine the core tech. The first step is to decide what the actual technology in focus is. A technology in this case is a physical tool, process or system of other technologies that combine to do a job. It does not include businesses or social innovations. (A “business” is just a group of physical and social technologies united under a strategy — but we’re only concerned with the physical part here.[1])

Examples can range from the simple: hammers, glass, newspapers, or an assembly line process; to the more complex: CPUs, streaming video services, blockchains, smartphones, or nuclear reactors.

Step 2: Layers below. What are the primary technologies and processes needed to create and maintain the core tech? What does it need not only to function but to be sustainable? Clues can be found in:

  • Why now: What enabled the tech in the first place? Why wasn’t it widely used 20 or 50 years earlier?
  • Suppliers & major cost centers of businesses producing the tech. (Infrastructure, manufacturing tech, service networks…)
  • Supply chain & logistics: What gets the product/service to customers? (Transportation, shipping, internet…)
  • Distribution tech: What gets the customers to the product? (Retailers, advertising, search engines…)

Step 3: Layers above. What does the tech directly enable? It’s possible there are no layers here. Many well-known innovations don’t directly enable other tech, like Netflix.

  • What do other businesses use it for? Who is it a supplier to?
  • Is there anything in-between the technology and the ultimate end-user?
  • Is it a marketplace or multi-sided network that connects many groups of users?

Stack tree examples

Here’s a few examples of stack trees from the tech industry, although they can be drawn out for products any industry:

(The Amazon “Vampire Squid” is the best example I can think of traversing the stack, starting as an online marketplace and expanding outward in all directions: up, down, and sideways (I left out Prime, Music, Video, etc.).

What insights can be gained?

Companies are embedded in value networks because their products generally are embedded, or nested hierarchically, as components within other products and eventually within end systems of use. — Clayton Christensen

A tech stack tree is one way of looking at a company’s value network. This can lead to insights into where value flows, who captures it, and potential opportunities to expand the business.

What layers in the stack capture the most value?

Which technologies accrue most of the value depend on many things: how much value (productivity) created relative to alternatives, availability of potential substitutes, the size of the overall need, or other competitive advantages inherent to the business model.

One of the models Clayton Christensen uses makes sense to apply here: Where is the bottleneck in demand? In other words, where in the stack is the biggest difference between performance demanded and performance supplied? What needs to be better?

Nvidia is a good example here. They keep churning out GPUs with higher capabilities and the market keeps needing more. Supply hasn’t kept up with demand and that’s likely to continue for some time. This bottleneck (along with other factors ) allows the GPU layer to capture a lot of value.

Are there opportunities to expand into adjacent technologies?

Amazon (see stack above) is the prototypical example here. They started as an online marketplace with some fulfillment operations, and over time have expanded in all directions.

In more traditional business thinking, you consider expanding vertically into suppliers and customers or horizontally across industries. Traversing a tech stack is similar, but to me more focused on the true technological and needs-based relationships. Traditional business thinking would have never led to Amazon expanding into internet infrastructure via AWS.

Of course, expanding for the sake of it is never a good strategy. You have to ask:

  • Do our current products or processes give us an advantage here?
  • How much value does the layer capture? (Is it a bottleneck in demand?)
  • Are there existing barriers to entry, and if so, does our position in other stack layers help overcome them?
  • Does this improve the outcomes for our current customers?
  • Will expansion endanger our relationships with partners or customers?

Short case study: Food delivery apps

The core tech here is a mobile or desktop app where you can order food from many local restaurants and get it delivered within ~1 hour. DoorDash, UberEats, Postmates, etc.

Layers below: What are their major cost centers? Restaurants and delivery drivers. What enabled delivery apps? Primarily ubiquitous smartphones and access to GPS-based navigation. Restaurants also need to have some way to communicate, whether by phone or Wifi-based tablets, and be able to package food in proper take-out containers (plus potentially many others to manage operations).

Layers above: What needs delivery apps to run? Cloud kitchens, which operate large strategically located kitchens that can make food for many different branded “restaurants”. Recently a further layer was added with the concept of pop-up branded chains, which uses the cloud kitchen & delivery infrastructure.

What captures the value? In the stack above, smartphones capture far more value than any other tech — but they’re a platform with thousands of other use cases. In this case we just want to focus on value flow within the food delivery market. It may not be clear at first who captures more value: the delivery apps or the restaurants, given companies like DoorDash are losing so much money. But it’s clear that restaurants are not a bottleneck in demand — so it’s likely the apps that capture more value. And it seems their unit economics bear this out.

Opportunities for expansion? The clearest opportunity to expand within the tech stack is into cloud kitchens. This could potentially alienate some restaurant partners, but restaurants are so fragmented it shouldn’t matter. I think this has a lot of potential given: captive customers, synergies with delivery app, and lower costs with economies of scale and not having to operate normal restaurant ops.

Functions in the stack

How would you classify technologies in the stack? I think it’s most informative to categorize by what pattern (or archetype) they match in the greater ecosystem. These are functions that can exist in any industry or stack tree: Platforms, protocols, etc.

I’ll follow up with another post including examples of different tech functions and stack patterns.

To be continued…


Thanks to Leo Polovets and Jake Singer for helpful feedback on this post. Header photo from veeterzy on Unsplash.


Footnotes

  1. Physical technologies are “methods and designs for transforming matter, energy, and information from one state into another in pursuit of [goals]“. There are also social technologies (organizational processes, culture, values, incentive systems, memes, etc.) that evolve and build off of each other over time. (Definitions from The Origin of Wealth, by Eric Beinhocker.) ↩︎

Build Series: Frameworks for Effort

In April, Marc Andreessen put out the call to build. It was in response to our failure to control and mitigate the effects of Covid-19 — institutions on every level were unprepared for the pandemic, and have continued to show their inability to quickly find and scale solutions.

But more than anything it was in response to our failure to build in general. We chose not to build, he claims. “You see it throughout Western life, and specifically throughout American life.” The problem isn’t a lack of resources or technical ability — it’s with supply and demand of desire. Demand is limited by our ambition and will to build. Supply is limited by the people and organizations holding it back.

Andreessen is generally an optimist, which is why I see his essay as positive in overall tone. But it was also somewhat of a mea culpa. Andreessen has for years been on the other side of Peter Thiel’s view of modern technical stagnation.

Thiel’s view may be too pessimistic, but there’s a kernel of truth to it. If you’re familiar with the history of tech and innovation, something feels different. The late-1800s to mid-1900s had explosions of innovation in fields from medicine to consumer products, transportation, energy, communication, computing, food, and more.1

This is the introduction to a series of ongoing essays centered around the question:

What frameworks can help us build more, better?

And further attempting to investigate the answers to the following:

  • What are the best ways to approach solving big, complex problems?
  • Why are certain efforts so much harder to achieve than others?
  • How are these efforts best managed at every level?
  • How do we build things faster? (Without sacrificing quality or safety.)
  • What is holding us back from building more?
  • How do we overcome these barriers?

Many of these lessons apply not just to “building” in the physical sense, but for solving problems, scientific discoveries, improving systems, and making progress overall. Building in a way is symbolic. It represents making big, necessary changes to move humanity and our planet forward. This can be building something physical or digital, pushing the boundaries of fundamental research, or trying new uncertain ways to solve problems.

It doesn’t even have to be anything new or innovative per se. Andreessen gives many examples of expanding existing tech: housing, infrastructure, education, manufacturing. Even preservation and restoration — in many ways opposites of building — can still apply. In the early 1900s, President Teddy Roosevelt established over 230 million acres of public lands and parks. This added an incalculable amount of value to future generations. I would love to see E.O. Wilson’s Half-Earth Project executed at scale. This is in the spirit of building: making progress and pushing humanity toward a better future.

Here’s a preview of some of the specific topics I want to explore in the series: Ladders of Abstraction (why/how chains), Oblique vs. direct approaches, Modes of effort (why/how quadrants), traversing fitness landscapes, the explore vs. exploit tradeoff, the role of trust in building things fast, forcing functions, and the specific methods we used to accomplish large-scale collaborative efforts such as the Apollo program, the Manhattan Project, etc.

Table of Contents

  • IntroBuild Series: Frameworks for Effort
  • Part I: Lay of the Land
    • Wayfinding Through the Web of Efforts [8 minutes] — Putting goals on a ladder or hierarchy of abstraction. Defining efforts and their multi-scale nature. Determining the hierarchy of efforts using a why/how chain. The difference between making progress directly and obliquely, and the consequences of misplaced directness.
    • Managing Modes of Effort [10 minutes] — A framework for understanding how managing progress differs across scales of effort. Classifying efforts into four modes on the how/what quadrants. Defining the modes and how they fit on the hierarchy of abstraction. A Covid-19 case study. How to manage an effort based on its mode.

Footnotes

  1. What was different about this era? The following is a good rundown from Vaclav Smil’s book “Creating the Twentieth Century” on the remarkable attributes of the pre-WWI technical era:
    • The impact of the late 19th and early 20th century advances was almost instantaneous, as their commercial adoption and widespread diffusion were very rapid. A great deal of useful scientific input that could be used to open some remarkable innovation gates was accumulating during the first half of the 19th century. But it was only after the mid-1860s when so many input parameters began to come together that a flow of new instructions surged through Western society.
    • The extraordinary concentration of a large number of scientific and technical advances.
    • The rate with which all kinds of innovations were promptly improved after their introduction—made more efficient, more convenient to use, less expensive, and hence available on truly mass scales.
    • The imagination and boldness of new proposals. So many of its inventors were eager to bring to life practical applications of devices and processes that seemed utterly impractical, even impossible, to so many of their contemporaries.
    • The epoch-making nature of these technical advances. Most of them are still with us not just as inconsequential survivors or marginal accoutrements from a bygone age but as the very foundation of modern civilizations. ↩︎

Pandemic Memo

The following are my thoughts taken from a memo to family office investors I sent out today regarding the pandemic.


These are unprecedented times in modern history. Not since World War II has there been such a large disruption in daily lives across the world at such a quick pace.

The pandemic we’ve entered is a classic Black Swan — an unpredicted event that has extreme consequences. Of course, Black Swan events are relative. A surprise to you or I may have been wholly anticipated by others. And in this case, it very much was.

To epidemiologists and people who had seriously thought it through, a global pandemic quickly sweeping humanity was an inevitability. It was a matter of when, not if. In 2018 Bill Gates gave a short TED Talk about the dangers of a global flu-like pandemic and the measures we could take to help prevent or reduce it. As we’re now aware, the advice was unheeded.

The human lives lost from the virus will be a tragedy of epic proportions. The current and upcoming economic malaise may be nearly as bad — particularly affecting those without the means to ride it out. Recent wide-ranging government stimulus and intervention can soften the blow, but ultimately the only solution is getting rid of the virus.

This is another reminder that we live on the thin veneer of civilization — modern society is very fragile if we’re not constantly vigilant about it.

We will get through this, as humanity has always done in the past. When the entire world has a common enemy, people get creative. Everyone should expect the world to look different after. Especially in areas like healthcare, biotech, and government.

These differences will all be for the better. Humanity is always searching for higher peaks of “fitness”, and on the rough landscape of possibilities sometimes you have to go down to eventually go up. Life getting worse before it gets better has always been a common theme. From the shift to agricultural societies, to world wars, to global pandemics.

We just need to work together to get through it first.

Advantage Flywheels

Competitive advantage can be represented visually as 1 or more feedback loops. These create the advantage “flywheel” that maintain and grow a moat over time. Think of a big, heavy wheel that takes some effort to get started but then coasts off its own momentum.

Before continuing, check out Eric Jorgenson’s primer on the flywheel mental model here.

Flywheel archetypes

Here are 6 simple examples of common advantages represented as flywheels (or “causal loops” in systems terminology). These loops are generalized — they’ll be expressed uniquely in every company that has them.

A few examples of how each advantage flywheel can vary:

archetypes.jpg
  • In the Economies of Scale flywheel above, the primary driver of more volume is low prices. This fits for most consumer businesses, but lower prices aren’t always the outcome of lower unit costs. If prices are maintained or increase, scale will yield higher margins → more resources to spend on growth → more sales volume.
  • The Brand Habit flywheel exhibits the typical loop for habit-reinforcing association of a brand with a specific quality or job-to-be-done. Think “thirst quenching happiness” for Coca-Cola and “low prices” for Wal-Mart. Another example of brand advantage is more of a social proof effect: Product has success → the cool kids want it → improved perception of product → …

As Eric discussed in his flywheel post, each wheel needs a push to get started. Written in green on a few of the archetypes above are initial advantages to get the wheels moving. Whether it’s a better user experience, a technical breakthrough, or a bootstrapped network based off of an existing network (college campuses for FB) or a useful utility (Instagram).

Real world examples

The above archetypes can be combined to create more comprehensive flywheels modeling the driving “engines” of each company’s moat:

examples

The most successful moats have multiple flywheels that feed off of each other’s momentum. Google’s technical advantages enable stronger brand allegiance and vice versa. Coca-Cola’s marketing-driven brand feeds off of it’s distributor/bottler based network effects. Facebook’s brands have at least 3 reinforcing network effects: direct (social network), 2-sided aggregator (advertising and developers), and brand-driven social proof.

Friction and limiting factors

In systems thinking, reinforcing feedback loops are almost always slowed by a balancing loop attached to it. Growth doesn’t continue unchecked, and flywheels always run into friction.

Some of these limiting factors are overcome, others are so strong they stop or reverse the entire growth engine.

What are some typical examples?

  • Switching costs & network effects — product quality slips as the incentives to improve aren’t strong when customers can’t leave → value of a competitive offering overcomes switching cost.
  • Learning curve of proprietary tech — hitting top of the S-curve, output efficiency declines, and competitors catch up.
  • Direct network effects — any source of decreasing value to users, which could cause users to exit and turn the virtuous cycle into a vicious one.

Moats Move

Using the analogy of a feedback loop helps to think of an advantage as a moving, changing system. A system that needs catalysts to get started, and will gain momentum at first but still be slowed by friction over time.

When thinking about how a business will grow over time, ask:

  • What advantage archetypes does it fit?
  • Where are the sources of positive feedback?
  • How do you get the flywheels moving? What strategies can help get inertia? (For example, “doing things that don’t scale.”)
  • What are the current or future limiting factors?

Featured photo from Ruth Hartnup on Flickr.
Thanks to Eric Jorgenson for feedback on the final version.

Polaroid, Apple’s spiritual successor

I just finished 2 books on the history of Polaroid 🌈1. A remarkable tech company with enormous success in consumer and industrial applications for decades. It’s also remarkable just how much Apple was influenced by Polaroid.

A brief history

As a child Edwin Land found a copy of the 1911 edition of Physical Optics, a textbook by the physicist Robert W. Wood. He obsessed over its contents, lingering on one chapter in particular: the polarization of light.

In 1928, Ed Land was 19 when he invented the first thin-sheet polarizer. He cofounded Land-Wheelwright Labs with a friend in 1932 after dropping out of Harvard. Their first products were polarized versions of headlights, sunglasses, etc.

They grew slowly with mostly small industrial contracts for 6 years, then reincorporated as Polaroid Corporation. During the war sales grew an order of magnitude, 80% of which went to the military for products like polarized goggles.

In 1943 Land came up with the idea for a film camera that can process right away instead of in a lab. R&D started immediately, but it wasn’t until 1948 their first camera, the Model 95, was released. It went on to sell 900k units in 5 years.

The 95 was a classic disruptive innovation: worse quality than traditional film cams, dismissed as not “real” photography, but appealing to a new market of customers. And profitable: camera for $90, film packages with 60% gross margins.

With all the new cash flow, they could plow it back into R&D. To Land, they had “. . . created an environment where a man was expected to sit and think for two years.”

Polaroid’s growth lasted decades longer, peaking in the ’80s right when, ironically, they won an historic years-long lawsuit against Kodak for patent infringement.

Apple, the spiritual successor

Poloroid-Apple.jpg

Back to the Apple comparison. The similarities are clear: from values, to marketing, to org structure, to product launches and demos.

Just like Jobs, Land was at the top of every invisible organizational chart. An anonymous former colleague: “Don’t kid yourself, Polaroid is a one-man company.”

When faced with scientific illiteracy or lack of imagination, Land resorted to a restrained bit of showbiz. As it turned out, he was strikingly good at explaining his work to people, and powerfully persuasive.

Ed Land was one of Jobs’ childhood heroes. Jobs met with him later and connected when when Land said his products have always existed, they were just invisible: waiting to be discovered. Apple exemplified Land’s motto “Don’t do anything that someone else can do.

Polaroid’s downfall started long before the digital apocalypse with their sidelining of Land in the ’80s. His final mistake was giving little thought to his own succession and the future of the company in the new generation. When they all but kicked Land out, Jobs met with and scolded management, saying Polaroid would turn into “a vanilla corporation”.

And it did. Jobs would take this lesson to heart many years later with his own succession plan.

Snapshot

Evan Spiegel is also heavily influenced by Land and Polaroid. But alas, Snap is not a camera company—they’re a communication company. And I think they’d do better in the future remembering that.

Inspiration, not imitation.

snap.jpg
Polaroid Variable Day Glasses; Snap Glasses.

I’ll finish with a Land quote from 1970: “We are still a long way from the… camera that would be, oh, like the telephone: something that you use all day long … a camera that you would use as often as your pencil or your eyeglasses.”


  1. Instant: The Story of Polaroid” by Christopher Bonanos (2012). “Land’s Polaroid: A company and the man who invented it” by Peter Wensberg (1987) ↩︎

Tokenized Securities and the Future of Ownership

In the coming years, Tokenized Securities are poised to take over existing financial markets and create many where they didn’t exist before. This is only now possible due to the invention of decentralized blockchains along with the recent influx of interest and capital.

So what are they? Here are a few good resources to start with:

token_types.png
A breakdown of token types from The Token Handbook. Tokens will have many uses but I think the biggest will be on the Securities and Asset side — not currencies as many believe.

There’s plenty of related buzzwords like blockchain, crypto, ICOs, colored coins, etc., but forget all of those for now. Tokenized Securities are digitized, programmable ownership. Legal ownership requires enforceable scarcity. Normally anything digital isn’t scarce, but they can be thanks to decentralized ledgers (blockchains).

Continue reading “Tokenized Securities and the Future of Ownership”

Product Study: Falcon 9

Last week I was outside of Vandenberg Air Force Base to watch the launch of SpaceX’s Falcon 9 rocket. (It was perfect weather and an amazing experience for my first launch!) To commemorate it, this is another one of a handful of product case studies I wrote to help understand successful product launches.

Falcon 9 was finished in early 2010, and had been in development since 2005. Its first flight occurred on June 4, 2010, a demonstration flight to orbit where it circled Earth over 300 times before reentry.

  • 1st flight to ISS: May 22, 2012
  • 1st cargo resupply (CRS-1): October 7, 2012
  • 1st successful commercial flight: September 29, 2013

Development costs for v1.0 were estimated at $300M. NASA estimated that under traditional cost-plus contracts costs would have been over $3.6B. Total combined costs for F9 and Dragon up to 2014 were ~$850M, $400M of that provided by NASA. 

By September 2013, the SpaceX production line was manufacturing 1 F9 every month.

(1) Value created — Simply describe the innovation. How did it create value? 

The Falcon 9 is a two-stage rocket that delivers payloads to Earth orbit or beyond. It’s a transportation vehicle to space. F9 drastically reduced launch costs, allowing NASA and small satellite companies to send payloads at a fraction of the cost.

(2) Value captured — Competitive advantages, barriers to entry. Why didn’t incumbents have a reason to fight them?

  • Ahead on the learning curve — highly advanced, experiential, expert knowledge
  • Capital and time barriers — lots of money and time needed to get to scale
  • F9 was a disruptive innovation, built from the ground up at low cost. Incumbent launch companies had no reason to start from scratch and lower their profits when they had strong (mainly cost-plus) contracts with existing customers. Industry was viewed as very inelastic and that little demand existed at low end.

Continue reading “Product Study: Falcon 9”

Product Study: iPhone

One of a handful of product case studies I wrote last year to help understand successful product launches.

Apple’s iPhone was announced December 9, 2007 and released June 29, 2007. It was $499 for the 4GB version, $599 for 8GB. After 8 years it had captured 50% of U.S. smartphone market and >66% of sales, with 100 million users.

(1) Value created — Simply describe the innovation. How did it create value?

The iPhone is a pocket computer. It has typical phone capabilities including phone calls and text messaging, along with cellular internet connectivity. Differences between other smartphones at the time were:

  • Large multi-touch screen with no tactile keyboard, no need for stylus — this allowed full use of screen when not using keyboard
  • Ability to browse normal, non WAP, websites (can zoom easily using multi-touch)
  • Ability to run desktop-class applications
  • Multiple sensor inputs — proximity, light, accelerometer

(2) Value captured — Competitive advantages, barriers to entry. Why didn’t incumbents have a reason to fight them?

  • Distribution:
    • Extension from existing Apple network — iTunes, Mac OS, iPod.
    • Brand attachment to Apple.
    • Economies of scale exist with integration and complexity of engineering.
  • Switching costs once owning an iPhone.
  • Strong habit attached to usage many times / day — strong attachment to UX.
  • Phone makers saw it as toy for rich people at first. Computer makers didn’t see it as a computer (low-end disruption).

Continue reading “Product Study: iPhone”

Mashgin: The Future of Computer Vision

twitter-picAbout a year ago I invested in and joined a startup called Mashgin. In this post I want to talk a little about what we’re working on.

Mashgin is building a self-checkout kiosk that uses cameras to recognize multiple items at once without needing barcodes.

The current version of the kiosk is designed for cafeterias, where customers can slide their tray up and everything on it is recognized instantly. Excluding payment, the process takes around 2 seconds. No more waiting for a single line held up by a price check!

But retail checkout is just a package around Mashgin’s core fundamental technology. We believe there is an opportunity to apply recent technical advancements to many fields. Advancements such as:

  • Smartphone dividends — cheap sensors and ubiquitous, miniaturized electronic components
  • Cheap parallel processing power including low-cost GPUs
  • An explosion in collaborative, open-source software tools
  • Machine learning methods, in particular convolutional neural networks (a byproduct of the 2 preceding trends)
  • Cheap cloud infrastructure

Chris Dixon talks more about some of these trends in his post What’s Next in Computing?

So how is Mashgin applying this technology?

Adaptive Visual Automation

IMG_0330
Face swap: billionaire edition

Computer Vision transforms images into usable data (descriptions) using software. If cameras are the “eyes” of a machine, computer vision would be the brain’s visual cortex–processing and making sense of what it sees.

When computers know what they’re looking at, it opens up a world of potential. You can see it in existing use cases from facial recognition in Facebook photos (…or face swap apps) to Google Image Search and OCR. Newer, much more sophisticated applications include driverless cars, autonomous drones, and augmented reality.

Gradient Descent
A visual example of using gradient descent (the reverse of hill climbing in a fitness landscape) as part of the learning process of a neural network

These recent applications tend to be more complex, and as a result use machine learning in addition to traditional image processing methods. Machine learning, and in particular deep learning through neural networks, has changed the game in many areas of computer science, and we are just beginning to see its potential. ML can simplify a large amount of data into a single algorithm. As the name implies, it can learn and adapt to new information over time with little or no “teaching” from engineers.

Both CV and ML can be applied to many fields, but one of the biggest immediate needs is in Automation. There are a surprising amount of simple (to humans) visual tasks ripe for automation. This includes industrial use cases in manufacturing and distribution, and consumer use cases in household robotics and relief of everyday bottlenecks.

I call the above combination adaptive visual automation: using machine learning to automate vision-based tasks. Although relatively new, this combination covers a large and quickly growing class of real-world problems. Autonomous cars (and especially trucks) are a good up-and-coming example that will have huge ramifications.

Mashgin’s future

Mashgin uses adaptive visual automation to improve the speed, accuracy and cost of applications in recognition, measurement, and counting in a closed environment. That was a bit of a mouthful, so here’s the short version: Mashgin wants to make visual automation intelligent.

There’s a broader category of AI vision companies whose purpose is giving computers the ability to understand what they see. Mashgin is a subset of this group, focusing on automating well defined real-world problems.

There are further subsets such as eliminating bottlenecks in everyday circumstances — speeding up checkout lines being one example. In many of the activities you do on a daily basis, intelligent automation has the ability to save a huge amount of time and money.

Retail checkout is a big market (even for just cafeterias) but it only scratches the surface of the value Mashgin will eventually be capable of. We have already established a foundation for applying recent advancements to these problems and it will only get better from here.

Atlastory: Mapping the history of the world

Certain ideas are “inevitable” over time. Paul Graham calls them “[squares] in the periodic table” — if they don’t exist now, they’ll be created shortly. It’s only a matter of when, not if.

I believe that Atlastory is one of those ideas. The following is a long post about a project I’ve been passionate about for some time now and am currently in the process of winding down.

The Idea

Atlastory is an open source project to create an interactive map that chronicles the history of life on earth. It’s a “Google Maps” for history. The ultimate goal is the ability to see what the world looked like 50, 200, 1000+ years ago. It was inspired by OpenStreetMap & Wikipedia: combining historic maps with cultural & statistical data.

Atlastory map in action

I started Atlastory at first because I’m a fan of both history and good data visualizations. I was surprised something like this didn’t already exist and thought that it would be an amazing educational tool.

Maps are one of the best ways to clearly show an enormous amount of information. Since everything in the past took place at a certain time and location, maps are an obvious choice to visualize that knowledge. Understanding history requires seeing changes and interactions over time, and a four-dimensional map allows this.

To envision information—and what bright and splendid visions can result—is to work at the intersection of image, word, number, art.” — Edward Tufte

Good design will be a key aspect of the final product. Good information design can communicate a huge amount of knowledge in a small window of time or space. Great information design has a high amount of density and complexity while remaining completely understandable.

The Vision (version ∞)

Atlastory’s purpose is to improve understanding of the past by organizing and visualizing historic knowledge.

My vision for Atlastory was that one day it would become a tool like Wikipedia that’s used regularly around the world. A journalist could use it to go back 20 years to see the geography and timeline of a major world event. A student could use it to go back 20,000 years to see the expansion of human culture across the globe. A climatologist could use it to visualize the historic overlap of population growth with changes in global climate patterns.

Wikipedia organizes information by creating a searchable network of interconnected articles that combine text and other multimedia. Atlastory can be the first medium that allows completely visual navigation, displaying information at a much higher density and level of interactivity.

1937-WORLD

Imagine students in a classroom learning about World War II. You’d be able to see the country borders of Europe as they existed in 1942. Drag the timeline, and see the borders change as the years go on. Turn on an overlay of population density or GDP per capita and see the flow of activity throughout the war. Zoom in and see the troop movements of a pivotal battle.

The visual interactivity would make it much more enticing for people, young and old. Almost game-like in terms of exploration and discovery.

Eventually, the timeline could go back far enough that you’re able to see continental drift and other pre-historic geographic or environmental changes.

Map content

Maps can be broken down into a few different types:

  • Physical — shows the physical landscape including mountains, rivers, lakes.
  • Political — sovereign, national and state boundaries, with cities of all sizes. The typical world map you see will be political with some physical features.
  • Road — shows roads of various sizes along with destinations and points of interest. Google Maps & other navigation apps fall into this category.
  • Statistical — shows statistics about human populations such as economic stats, population density, etc.
  • Scientific — thematic maps that can show climate, ecological regions, etc. (see the climate map below)
  • Events — shows how a specific event played out geographically, like WWII or Alexander the Great’s conquests.

Climate patterns

Any map type that has enough data to span long periods could eventually go into the Atlastory system. Event, thematic, statistical, and scientific maps could all seamlessly layer on top of the main “base map”.

Base map

The Atlastory base map should be an elegant combination between 3 map types: physical (basic landscape features), political (sovereign and administrative boundaries), and cultural (see below). Major roads and infrastructure would be added only after a worldwide “structure” of the base map was created.

Importantly, map creation should be top down, from global to local. The purpose of an Atlastory map is not navigation, it is understanding of history. Creating a global structure will also provide context and make it easier to interest other users/contributors.

Cultural cartography

Most world maps made today (of the present time or of the last few hundred years or so) are of the political variety. But what happens when you go back a few thousand years? What about areas of the world where, even now, aren’t necessarily defined by geopolitical boundaries?

The solution is mapping cultural regions. Culture, in this case, being human societies with common language, belief systems, and norms. “A cultural boundary (also cultural border) in ethnology is a geographical boundary between two identifiable ethnic or ethno-linguistic cultures.”

A cultural map would have different levels, just like political maps: from dominant cultural macroregions to local divisions between subcultures or classes within a society (blue collar vs. white collar, etc.).

Combining cultural cartography with typical map types allows for a much better understanding of both modern and ancient history. Culture plays a major role in world events & limiting the map to only defined borders paints an inaccurate view of history.

Cultural regions

(Notice any overlap between cultural regions and the climate regions in the map above it?)

The Tech

The technical infrastructure behind Atlastory has a few basic components:

  1. A database of nodes (latitude/longitude points) organized into shapes, layers, types, and time periods.
  2. An API that manages, imports and exports data from the database.
  3. crowdsourced map editor interface (like iD for OpenStreetMap, but designed specifically for top-down time-based editing).
  4. A map rendering service that turns raw map data from the database into vector tiles that can be styled for viewing.
  5. The map itself: a web interface to view and navigate the maps.

Most of the components would be built from existing open-source tools created by organizations like OpenStreetMap, MapBox, and CartoDB. There has been a lot of technical innovation in this field over the past few years which is one of the main reasons something like Atlastory is now possible to build. (Although given what I known about the requirements still very challenging.)

Read more about the technical requirements…

The current status and future of Atlastory

I’ve been working on this as a side project for more than 3 years now. Originally I imagined being able to quickly find a way to profit from the service. But as development dragged on and other commitments began taking up more of my time, I realized I’d never be able to finish it alone.

Earlier this year I joined Mashgin, a startup in the Bay Area, as a full-time “Generalist.” My spare time completely dried up and I decided everything needed to be completely open sourced and distributed to anyone interested in the project.

Due to personal time constraints, I can’t continue with it so I’m looking for others who are interested. This could mean taking over / adapting the codebase or using other means to pursue the idea. See below for more details on what’s currently done. Although many of the back-end components are functional, the infrastructure is in a rather unusable state right now.

Please contact me or leave a comment below if this strikes your curiosity or you know anyone else who would be interested. I’m happy to answer any questions.

Resources