The Exploration Instinct

Res Extensa #63 :: How we’re wired to search for novelty

In a recent episode of the Complexity podcast, computer science professor Melanie Moses talked about her work on how systems of collective intelligence form and interact with their environments.

During her research, she spent time in the New Mexico desert observing colonies of desert harvester ants, studying how they communicate and make decisions.

Ants communicate by laying down trails of pheromones: chemical signals that attract other members of the colony. When it's time to forage, the group disperses, and when one ant finds anything, it’ll leave a pheromone trail from the source back to the colony. As more ants show up and follow the path, they leave their own chemical trail, too, amplifying the self-reinforcing signal toward the food source. Once the source runs out, the chemical signal eventually evaporates. It’s a clever system for passive, chemical communication. Search, notify, exploit. Repeat with all hands on deck until the resource is depleted.

But what actually happens is weirder.

Moses ran an experiment. She placed a large pile of 256 seeds close to a colony to see how the ants would react. The team would watch each ant and track whether they’d pick up the trail and help cash in on the seed source, or not:

One ant discovered it very early on within a couple of minutes of placing it. And that one ant, it went back and forth from the seeds to the nest and the seeds to the nest 256 times all by herself. And the other ants just wandered around completely not helping her. And so, you know, this is my model for incredible collective behavior. And they're just, they've left her all on her own to collect these seeds. They weren't finding much. They were wandering.

With food in short supply, you’d think the collective would mine the food source until depleted. But ant biology is tuned to force them to choose: exploit the known, or continue searching the unknown? Exploit this food source? Or keep looking for the next one?

Ants are wired to explore, even when exploration looks unwise. Bias to search has been selected as beneficial for the collective. The gains outweigh the costs.

This teaches us something about our own collectives — organizations, teams, companies —  and how they function.

Asymmetry of options

Ants spend a huge amount of time foraging for food. Food is scarce, valuable, and one would think the sensible approach would be to spare no expense harvesting a source until it’s depleted. Take advantage of what you’ve found. But even when one is discovered, ant neural chemistry is still calibrated to devote a large fraction of time searching for the next one. What’s going on here?

It’s about asymmetry, specifically the asymmetry of the upsides and downsides. Capping known, or predictable risk, but enabling future reward.

Exploiting the known resource is downside-preventing: capitalize on the resource while you’ve got it. As long as the one ant keeps dragging seeds from the pile back home, even one-by-one, the risk is understood. The worst-case scenario is the one ant takes too long and rain washes away the seeds, or a competitor finds the stash, and the remaining seeds go uncollected. Not great, but a known risk.

On the flip side, exploration is upside-maximizing. One ant exploits the seed stash, but the colony still devotes hundreds of others to the search for new sources. The upside of what they could find is nearly limitless. Nature is adapted to make asymmetric bets.

The human instinct

Like the harvester ants, we’ve got our own intrinsic instinct to explore.

Hunter gatherer societies of the neolithic were always on the move in search of territory and resources. During the Age of Discovery, European explorers spent untold resources sailing the globe in search of unknown, but hoped-for future, exploits. Even in our own individual lives — gloriously rich with exploitable resources by historical standards — we’re not quite satisfied with playing it safe and exploiting what’s close-by. There’s a pressure to explore, to search for novelty.

It turns out this instinct toward curiosity was naturally-selected, and built into your genes. This adaptive characteristic helped us diversify our diets and gene pools, find new territory for our growing populations, and enhanced our resilience to shifting environmental pressures like blights, droughts, and natural disasters.

As we’ve climbed Maslow’s hierarchy, some of these needs sound primitive. Our exploration instinct doesn’t have us spend much time hunting for food these days. But problems are inevitable, and we’re always encountering new ones. We solve them through the creation of new knowledge, and we only find that knowledge by exploring the unknown.

We’re naturally inclined to weigh upside and downside risks, versus excessively favoring one over the other. If all we do is search and never settle, we’re taking radically too much risk to survive — and survival is step 1 to perpetuating ourselves. But if we settle for stasis the moment we find a new innovation, a new resource cache, or a comfortable place to live, we’re on borrowed time until the next inevitable problem. We need to explore to find the next solution to the next problem.

Organizations

Ants are a “eusocial” species, meaning they’ve evolved the ability to form massive hierarchical, socially-coordinated groups that exhibit a collective intelligence — an emergent group-level behavior, as if the real organism is the “colony” and each ant is a dumb individual cell, useless on its own.

We’ve evolved our own version of eusociality. Humans are the only species that can cooperate on an infinite scale, outside of our own gene pool. We form ourselves into superorganisms to share resources and create knowledge.

On the everyday scale, we create companies and teams to marshal resources for both exploitation and exploration, to extend the reach of our exploratory capabilities in ways we couldn’t do alone. Thinking about this in the risk/reward framework, the more limited in resources, knowledge, or effectiveness your team, the less your opportunity for novelty search. We form organizations to create for ourselves economies of scale and divisions of labor that allow us the leeway to send “explorers” out on the hunt for innovation. For solutions to novel problems.

The desire to explore isn’t just a weird tendency humans have to be curious. It’s a trait adapted to manage risks and maximize future potential. When your team or your company is acting too risk-averse, hanging onto the past, milking only old discoveries, remember that exploitation isn’t the only way to mitigate risk. Stasis is its own existential crisis. Without solving new problems, entropy will defeat us.

If the ants spend all their attention harvesting the known seed caches, they’ll never find the critical new ones.

---

Thank you for reading. If you found this useful or interesting, please forward on to your friends and colleagues.