Monthly Reading, September 2022
Res Extensa #24 :: Building living systems, YouTube and tacit knowledge, long-term plans, learning from the East India Company, and the paradox of scale
This week we're riding out Hurricane Ian here on Florida's west coast. So far the storm is projected to make landfall south of us a good ways. Hoping for the best for those folks on the southwest coast.
On to the links — some great stuff this month.
There's a lot to unload here in this piece from Gordon Brander.
A COMPLEX SYSTEM THAT WORKS IS INVARIABLY FOUND TO HAVE EVOLVED FROM A SIMPLE SYSTEM THAT WORKED — The parallel proposition also appears to be true: A COMPLEX SYSTEM DESIGNED FROM SCRATCH NEVER WORKS AND CANNOT BE MADE TO WORK. YOU HAVE TO START OVER, BEGINNING WITH A WORKING SIMPLE SYSTEM
Gall's Law tells us that "A complex system that works evolved from a simple system that worked."
If your goal is to solve a complex problem — building a product, writing a mathematical proof, conducting scientific experiments, anything — you must start from a foundation of doing simple things first. Simple things that work. Simple seeds are the source of all interesting, living, complex systems.
Often a complex system arrives over years of incremental improvement, with no single person fully comprehending how the system's fitness developed over time. Take a complex system like an international logistics operation — our shipping networks evolved gradually over centuries, starting from a simple solution of "put this on the boat and take it over there." Now we've got containerships, massive port facilities, complicated tariffs, import regulations, bills of lading, hundreds of organizations involved. The system became complex after thousands of incremental adaptations, employed through interfacing with reality, each component developing in response to a purpose. But if you were to ground-up redesign a system like this, thousands of those micro-adaptations would be missed or underappreciated.
They call this the "second-system effect", from Fred Brooks's Mythical Man-Month:
The tendency of small, elegant, and successful systems, to be succeeded by over-engineered, bloated systems, due to inflated expectations and overconfidence.
There's a similarity to Chesterton's fence here. Just. because you don't know the purpose of an individual element of a system doesn't imply that it's purposeless. Some knowledge is simply out of reach.
Thanks for reading Res Extensa. Subscribe for free to receive new posts and support my work.
You can broadly categorize knowledge into 2 categories: epistemic vs. experiential, or theory vs. practice. Samo Burja has an interesting theory on how YouTube is contributing to the transfer of practical tacit knowledge:
Tacit knowledge is knowledge that can’t properly be transmitted via verbal or written instruction, like the ability to create great art or assess a startup. This tacit knowledge is a form of intellectual dark matter, pervading society in a million ways, some of them trivial, some of them vital. Examples include woodworking, metalworking, housekeeping, cooking, dancing, amateur public speaking, assembly line oversight, rapid problem-solving, and heart surgery.
Before video became available at scale, tacit knowledge had to be transmitted in person, so that the learner could closely observe the knowledge in action and learn in real time — skilled metalworking, for example, is impossible to teach from a textbook. Because of this intensely local nature, it presents a uniquely strong succession problem: if a master woodworker fails to transmit his tacit knowledge to the few apprentices in his shop, the knowledge is lost forever, even if he’s written books about it.
Tacit knowledge resists attempts at centralization. Academic institutions have long specialized in the epistemic: knowledge that can more easily be transferred through reading and writing. Tacit knowledge is distributed and local, requiring physical presence with a master craftsman to observe in detail and learn technique. For centuries (until the information revolution) nearly all useful skills were practical. You became a blacksmith, cooper, carpenter, or haberdasher through apprenticeship. YouTube has created an aggregation platform that creates an online master-apprentice relationship, and provides an avenue for communicating thousands of practical skills.
Long-term plans in product development are a conundrum. We need to have some sense of where we're headed, but we actually have no way of knowing all the details of how to get there when we start. A common trope in software is to look to manufacturing systems for guidance. People love to look to principles like continuous flow, kanban, and kaizen: all great reference points we can learn from and adapt to work on software products. But as Lucas Costa points out here, comparisons to manufacturing can break down if applied to comprehensively. Primarily because in a production line, the design is complete before raw materials enter the facility. In software we don't have this constraint (to be forced to fully design for every detail up front), so the design can evolve along with the building process. Makers of physical products would love to have this adaptability, and concepts like JIT push in that direction.
Lucas Costa calls long-term plans "his favorite genre of fiction". He points out that the commingling of designing and building confounds the forecasting process when making software:
In the world of software development, manufacturing is not a challenge. In fact, replicating and distributing software is virtually free.
The problem with software development is that we try to forecast how long it will take for us to design features, not to replicate them. Because every new feature is — obviously — new, there’s just no way of knowing how long feature development will take unless we actually do it.
Through a lens of effectiveness vs. efficiency, design is concerned with the former, production with the latter. "Product risk" is a design problem, "operational risk" is a production one. "Can we build and deliver this cost effectively?" is a relevant question, but I think it's downstream of "Does anyone want what we're making, anyway?":
Moreover, while manufacturing poses operational risks, design poses product risks. When manufacturing is poorly managed, you have fewer products to sell or narrower profit margins, but you still have a product people are willing to buy because you’re past the design stage. On the other hand, when a product is poorly designed, it doesn’t matter how efficient its manufacturing and distribution are because no one is willing to buy it.
You can’t predict how long it will take to act on your customer’s feedback. Hence long-term plans don’t allow space for working on feedback. They don’t do so because feedback adds variability and uncertainty. In other words, feedback corrupts the beautiful, deterministic, omniscient plan.
The East India Company was one of the first major international corporations. At one point in the mid-1700s, about half of the world's trade flowed through the EIC. Rohit Krishnan reflects on some interesting facts about its operations throughout its 270-year history. The most fascinating aspects have to do with the EIC's organizational design — how you coordinate an organization of 100,000+ staff spread around the globe in an age where communication latency between hemispheres was months and months:
The reason looking back at this era is so interesting is because communication latencies make almost any remote management impossible. Which means we get to see the benefits (and drawbacks) of having and needing high agency people to basically do anything with minimal oversight.
Decentralization of command-and-control was the only way to operate a company at this scale, at this time. Even with email and Zoom and single-day international travel, it's challenging to centralize the operational decision-making of a global organization. There's been a force of centralization that's inevitably reduced autonomy of distributed teams, and perhaps the costs of that reduction aren't always expected or wanted:
We take it as an article of faith that the higher levels of information transparency and speed is a good thing. And it indisputably is. But a consequence of it happening has been that in many (most?) cases it’s been used as a reason to reduce the autonomy at the edge.
Simon Sarris has been documenting the process of building his own home. From selection of land, to the deep thoughtfulness of the design and architecture, down to individual materials and hardware. As the title suggests, his goal is to design a house that revives elements that were historically common, but less so in modern McMansions. I've enjoyed the whole series so far.
I wrote recently on the blog about a paradox with building "scalable" companies, or software.
So much of what’s required to actually scale to Google or Netflix level is fundamentally unknown and actually nonexistent when they ran into their scaling frictions. Due to their unique needs to deliver their products to hundreds of millions of simultaneous users, Netflix builds Chaos Monkey, Google creates Kubernetes. There’s nothing wrong with these tools; they solve problems that are nearly one-of-a-kind for businesses that are in a class of very few.
You have to know when to prepare for scale and how much. The formative early days of a company need all of your attention on your core innovation, your core product-market fit. Unfortunately the industry promotes too much binary thinking on best practices: "you should do this, or else".
Thanks for reading Res Extensa! Subscribe for free to receive new posts and support my work.