# Navigating

*We proceed without certainty*

From "Field Notes from a Centaur" by Michell Zappa. Canonical: https://www.michellzappa.com/field-notes/navigating
Cite as: Michell Zappa, "Navigating", Field Notes from a Centaur, https://www.michellzappa.com/field-notes/navigating

---

I've been looking at the evolution of AI from the perspective of augmentation for as long as I can remember. Some of these ideas are a decade old, others are new. Since 2024 I have given them as a talk in ten parts, and the examples change every time. The ten parts have stayed put. I haven't left this framework behind yet. I think it still works.

It is a framework for non-experts, to level up their understanding of both the theoretical side of AI and the practical ways of using it, without going into how it works. How it works is its own thing. You can spend your whole life on that. We are primarily users of it, not builders of it. I'm in the same spot: not very technical, but with some understanding. And I think we can grow our technical understanding by asking the right questions and spending the time with the right tools.

The idea is that we start with the underpinnings of technology, which is exponential change, and we build up from there to understand what might be possible if we reach artificial general intelligence, and what that implies.

## Exponentials

As a technologist who's asked to show up and talk about where technology is heading, one of the biggest certainties is that everything is accelerating when it comes to information technology. You have a doubling of performance or a halving of cost every 18 to 24 months. Moore's law is one name associated with this, but there are many other power laws that mean the capacity of our information systems, so storage, bandwidth, memory, processing, keeps going up and to the right, and the equivalent cost keeps going down. This has been true for many, many decades.

If you take this and extrapolate, you get a really good sense for what will be possible a few years down the line. This is arguably how all tech companies think and operate. It slows down at times, but the general principle is this will keep going. There's no reason to believe this is going to stop or taper off.

There's a curve I've been using for a lifetime, way before any of this. We tend to overestimate the short-term effects of technology while we underestimate their long-term effects. Roy Amara put it that way, and as a metaphor it has always worked. Our expectations of the world are linear. The world, in technological terms, tends to be exponential, and exponentials are very, very difficult to wrap your head around.

![Amara's law: we overestimate the short term and underestimate the long term](/images/navigating-ai/amara.svg)

With AI it means we have to keep updating our expectations of how things actually work. Image models used to botch hands, and everyone made fun of it. Ask any image model for hands now and they're fine. I call the inability to see this exponential slope blindness.

Technology also has a direction. It moves towards lower costs, but also towards more ubiquity and increased complexity. And it changes our expectations. We used to hunt when we were hungry, now we tap. There's no way of going back, not even a little step. This is sometimes called technological somnambulism (Langdon Winner's term): sleepwalking, the idea that we can pick up a technology and then put it back down and somehow be unaffected by it. But that's not true. The moment we've used a technology, it has changed our perception of the world, and we have changed. It's like a ratchet, those cogs that only move in one direction. You can't twist it back.

AI is a so-called GPT. Not the generative pre-trained transformer, but the other GPT: a general-purpose technology, one that can be used for much more than what it was invented for. These technologies become invisible. The printing press is just there. We can't even imagine the world without electricity. What do you mean there's no phone, what do you mean there's no internet? It wasn't here fifteen years ago, and now that's how the world is supposed to work.

AI also sits at a very interesting intersection. It's a provocative statement, but there are no experts on AI. Nobody can be an expert on all of it, because it sits between areas of research that each take a whole life to get into: computer science, statistics, linguistics, psychology, data science, ethics, math and many more. It turns out the only one who knows all of these is the language model, which is a very interesting twist. The rest of us are limited to one point of view, which is why I like coming at it as a generalist who has no depth in any of these areas, but enough to see a big picture.

## Frontiers

There are many kinds of AI. I have a rice cooker, and my rice cooker says somewhere on the box that it uses AI. First you laugh: okay, that's silly. But then you're like, no, that's actually it. By definition it's a reactive machine. It's the same principle that is used for spam filters and even chess computers. It's utterly distinct from generative AI, which is in itself distinct from predictive AI, the kind companies use in supply chains.

It's crucial to keep an eye on the state of the art, because it's changing every single day. The frontier models of six months ago are not the ones you should judge by, and reading about them is not enough. You need a mental picture of how good these systems actually are, and the only way to get it is to spend time with them yourself.

That picture is harder to form than it sounds. We expect AI to be really smart sometimes, but it isn't. Voice assistants are sold as really smart, and they're pretty crummy. The opposite of that is when ChatGPT came about, or when AlphaGo (DeepMind's Go-playing system) won against Lee Sedol: actual performance way outweighs our expected performance. We think things are going to be so-so, and then they're like, whoa. A study by BCG with Harvard researchers called this the jagged frontier. I don't like the consultancy habit of the two-by-two matrix, but the underlying idea is solid. It's really difficult to know where these systems perform well and where they don't. It's completely unpredictable until you've spent quite a bit of time using them.

![The jagged frontier: expected against actual performance](/images/navigating-ai/jagged-frontier.svg)

Image models are the easiest place to watch the frontier move. FLUX was my favorite for a long time, and its pictures will fool virtually everyone. In video, the capabilities keep going up. There was the infamous clip of Will Smith eating spaghetti, painful to look at in its amorphous shape, and a year later you could not tell the clips apart from footage. There are now so many AI-generated videos that we're no longer seeing them.

Robotics hardware has been getting better for years, but the software has been pretty limited. Most robots are pre-programmed. They can't discover how to do new things, whereas what we expect from robots is that they should be able to learn, to pick things up from us. With language models that is starting to happen, which is why nearly every AI lab now has a robotics project.

The frontier has since moved from pictures to work. METR, a research group that measures what AI agents can do, found that the length of the tasks agents can finish on their own doubles about every seven months. Whatever you saw six months ago is no longer the edge.

## Centaurs

There's a metaphor I've been using for about a decade now: the idea of augmentation as opposed to replacement. AI is often used in the sense of, let's replace people, functions, jobs, tasks with them. That's certainly one way of using them, but there's a whole other way, which is an awesome story in my opinion.

![A centaur combines human skills with machine qualities](/images/navigating-ai/centaur-skills.svg)

Kasparov played against IBM's Deep Blue supercomputer, and IBM won. The match was characterized very much as one versus the other, human versus computer, and that's the story that we took away. When the computer becomes better than the human player, why should the human player even practice anymore? Kasparov decided to do something different. Instead of practicing against the computer, he said: let's use computers and create a new category of chess. It's now known as freestyle chess, in which a player with a computer plays against another human-computer pair. The first championship happened in León, in Spain. Kasparov says it was as disturbing as it was exciting.

For about two decades, the best players in the world in so-called blind matches, where you don't see your opponent, have been centaurs. They outperform grandmasters and supercomputers regularly. On the Elo rankings, the very best human players score around 2,900 and the best chess programs around 3,500. Above that it gets fuzzy: some years the engines win, other years the centaurs. What stays true is that the process is what makes you win. It's not having the best computer. It's not knowing the game the best. It's knowing your machine better than your opponent.

Think of Neo downloading kung fu in The Matrix: now I can do all of these things I wasn't able to do before. In fiction it's a download. For us it looks very different. Netflix made a centaur decision when it bought House of Cards. It made casting and creative decisions, down to the director and the color palette, with data, which was utterly novel at the time. Their head of content described it as about two-thirds gut and one-third data. In football, the video assistant referee puts a few more eyes on the ball, with no AI involved at all. The slow introduction of other forms of intelligence is what I would characterize as centaur thinking. Researchers who built a model to restore missing text in ancient Greek inscriptions found that historians alone got about 25% right, the model 62%, and historians working with the model 72%.

The study with the best numbers is the one by BCG together with Wharton and Harvard, the same one that named the jagged frontier. If you're under average on any skill, you have the biggest potential gain: GPT-4 lifted below-average consultants by about 43%, up to above average. If you're already above average, you still get a boost, about 17%, but it's much smaller.

![GPT-4 lifted below-average consultants by 43 percent and above-average ones by 17 percent](/images/navigating-ai/skill-boost.svg)

The takeaway, to me, is that we're below average on most things. We're good at a few things and we specialize in those, but on everything else you have this massive opportunity to become above average on things you know nothing about. It's not so much "will it be better than us", but more "it's going to make everyone as good as everyone else". That's a complete reframe of how we can think about all of these systems. So don't ask AI to do what you already do well. It will probably be worse than you.

We can't be too reliant on these systems either. There were mushroom picking guides for sale that were generative. You really don't want that. You still need an expert in the loop to decide what's credible and what isn't. We have to always intervene and be the person in the loop.

We have many forms of symbiosis in nature. Some are mutually beneficial. In others you have one side helping the other. And then you have predatory relationships, where they don't even know they're in an ecosystem with us. So we have to care about what kind of relationship we want with our machines, what kind of centaurs we're actually building. Who are we, and who's the machine in this system, is a hard question.

## Prompts

There's a lot of data being fed into these models, and a lot of parameters that determine the outcome. Some of this we see. Most of it we don't. We only really notice the bit that we're writing. If you use custom instructions, you've seen that there's more going in than what's on screen, and if you've read about system prompts, the instructions the company writes before yours, you start understanding how much calibration goes on behind the scenes. The prompt you write is just the tip of the iceberg.

For the part we do see, it helps to add a lot of structure. That's a general rule I've found applies everywhere. The framework I teach has four parts: a persona, the role the model should take; a task, what it should do; context, the background and specifics that matter; and a format, what the output should look like. But the most important thing is to use any framework at all. The more detail you give the input, the better your output becomes. There's no doubt about it. Reasoning models do a little of that expansion for us, and even then you get better responses if you give them structure. What does your output look like? What's an example of what you need? What should it not be doing?

Language is hard. If you ask an image model for a salmon jumping up a river, you might not get what you meant. We have to be very specific. These systems do what we ask them to, which isn't always what we want them to, and that gap is where a lot of difficulty takes place. When a conversation goes the wrong way, edit an earlier prompt instead of arguing with it. The chat branches from that point.

Compare models. We get used to one tool, the one that holds most of our data or that we've grown accustomed to. I find it hugely valuable to compare different models all the time, two or three for images, same thing with text. Depending on the context, you start learning what some do better than others, and you can use one to fact-check another.

And know which questions to ask. These systems are incredibly knowledgeable. They know so much more than we do. Asking "what else should I know?" at the end of any conversation is such an easy cheat: what did we miss, what else should we be looking at? Nine times out of ten that's valuable. Having a good understanding of what the models are capable of takes work, and it gives a lot of return if we spend the time on it.

## Creativity

We used to think AI was going to take the quote-unquote easy jobs. Truck driving looks really easy on the surface: you just follow the road. It turns out it isn't easy at all. These expectations have flipped significantly. There's a tweet I think we've all seen: I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do my laundry and dishes. It's good at doing the things that we actually enjoy doing.

Niceaunties, an artist in Singapore, makes very imaginative work that would be impossible if it weren't generative. You could do it in Photoshop, I guess, but you wouldn't. Leaning into the generative aesthetic is one way. Designing your own image models is another. Not replicating, but augmenting, is the direction forward here as well.

My own example is the tarot. I always wanted to make my own deck, and I'm not an illustrator at all. When DALL-E 3 came out, for the first time you had an image model with a language model behind it. With Midjourney you have to be extremely specific, because it will do exactly what you ask it to. With DALL-E, everything you ask goes into a language model first, and that makes all the difference in the world. You can ask it to consider every existing tarot card for the Sun, take everything it knows, and come up with something unique that combines these symbols. I stuck to that prompt and only changed the card, over and over, for the 22 cards of the major arcana. It took a couple of days to create something unique in an artistic sense, and also symbolically correct. I could never have done this in a million years. Image models give you something, language models give you something else, and the intersection of them is its own thing.

The same experiment taught me something less pleasant. People sometimes say the results are getting worse, and as a heavy user I always felt they were exaggerating. Then I did a test. Six months apart, with literally the same prompt and the same model, the Hierophant card came out varied and rich the first time and borderline horrible the second. Every extra layer of generation makes a request costlier, so my hunch, and a few people seem to agree, is that models become less capable over time, either to make the upgrades feel bigger or to reduce cost. The latter is more provable. I did not believe it until I experienced it.

For organizations, the practical questions are copyright and misuse. Some services prohibit anything copyrighted. With open models like FLUX or Stable Diffusion you can do just about anything, and any model can be jailbroken.

## Code

Where we go after creativity, in my opinion, is towards programming, because programming is a form of language that can be formalized and validated. There are infinite poems, and no poem is right or wrong. Either code runs or it doesn't run, and if it doesn't run, you can work on it until it runs. Models trained on code can get so much better than on other forms of language, simply because we can validate that the code works. The consequence of that is that these models are extremely good at coding.

You can describe the app or page you want in natural language, and the model will build it. Ask for a calculator and there's a calculator, and then you can ask for whatever alterations you can imagine. I agree with Andrej Karpathy: the hottest new programming language is English. People now call it vibe coding, and it's where I'm spending virtually all my time.

I spent 40 years wanting to code. I'm exaggerating a little bit, but my whole life I wanted to code and never really cracked it. I never had the patience for it, to be honest. And then all of a sudden you can do so many cool things and build so many cool things. I'm not a coder, but I've been doing so much code simply from knowing how to ask the right questions in the right order, and knowing how to take a few steps back.

A non-developer can build a complete application with a language model. You need to understand about 20% of what is going on, and the model provides the other 80. Even when you get 99% of the way, the last 1% may need an experienced developer. Since 2025 I have taught this to around 500 people who had never written code. For most people it is the highest-return way to spend time with AI. The best exercise is to pick a problem you have and solve it.

## Agents

To me, the culmination of having a lot of competency in the form of code is agents, and agents is probably the biggest, fuzziest word in AI. What's an agent? It's a system capable of perceiving the environment it's in and reacting accordingly in order to achieve a goal. The crucial thing is they have a feedback loop: they observe the outside, and they try to achieve a goal by changing their behavior until they get where you, the agent owner, want.

![An agent perceives its environment, decides and acts, in a feedback loop](/images/navigating-ai/agent-loop.svg)

Picture two axes. One runs from deterministic, systems that work in one very clear way, to autonomous, systems that can do things in any way. The other runs from narrow, very specific tasks, to general, open-ended ones. A spreadsheet sits deterministic and narrow: it always does what you want it to, it's very predictable, and it's very specific in what it achieves. From there we move through agents to AGI. Agents are the stepping stone towards artificial intelligences that are able to achieve open-ended goals in a general context. If this is fuzzy, everyone agrees. But there seems to be a strong, strong, strong desire to move in this direction, which I think is worth paying attention to.

The degree to which jobs can be replaced is high and not going to decrease. Goldman Sachs estimated that office and admin and legal work score around 40% of being exposed to automation, and the average is about 25%. This will just go up. If you're in a position of hiring, you're certainly thinking about what you can do for $20 a month with GPT as opposed to a salary for a person, and the implications of that are just absolutely massive. Agents working with agents, talking to agents.

It's going to pop up in invisible places. If you order takeout at a drive-through in the US, you might be talking to an AI from a company called Presto, and you'd have no idea. Waymo, Alphabet's self-driving car company, drives more autonomous miles month over month, year over year. We have self-driving cars. They're not everywhere, but they're here.

## AGI

From agents we move towards general intelligence. AGI is a theory. It's a hunch, it's a belief, and it's something that we might be manifesting. There's no map between here and there. There are many attempts at defining it, and there's a lot of money at stake.

![Levels of AGI, from Google DeepMind](/images/navigating-ai/levels-of-agi.svg)

OpenAI's independence from Microsoft is tied to whether they achieve AGI, and in a contract it can't be open-ended. So they define it as autonomous systems that outperform humans at economically valuable work. A lot of people have their own pet definitions of what AGI should be, but here is one that is contractually tied to a specific outcome. So you bet that this is what they're working towards: replacing labor.

Joscha Bach, a phenomenal philosopher of technology, describes intelligence as a series of vectors, among them knowledge, language, learning, reasoning, perception, creativity and autonomy. For AGI to be generally intelligent for real, it needs to be competent on all of them. Current models perform really well on learning and knowledge, and not as well on the others.

What I'm certain of is that AGI, if we are to see it, will be a combination of multiple forms of intelligence, including forms we're not seeing yet. AGI won't be language models. They will be part of it. An AI that can lean into other AIs, and knows when to talk to another AI and how, and can do that at the speed of light, will seem generally intelligent, in my opinion. And it will be.

Whether we go beyond human capability or not is arguably the biggest question out there. If these systems stay at our level, that is revolutionary in its own right. If they outperform all of us, that is so-called superintelligence, and it's anyone's guess what that might mean. It's a singularity: we can't see what's on the other side. Metaculus, a forecasting site where a few thousand people vote, used to put AGI around 2045, and the date keeps coming down. Do the math on how old you'll be.

The paper that literally sparked my own desire to write a newsletter is Sparks of AGI. Its first author, Sébastien Bubeck, argues that GPT-4 was already capable of thinking beyond its instructions, and that is a spark of AGI. And some companies already work very AGI-like. Ant Group, in China, serves around a billion users, a billion with a b, with ten to fifteen thousand employees. Its credit decisions use no human intervention at all.

Edsger Dijkstra said the question of whether machines can think is about as relevant as the question of whether submarines can swim. We don't ask ourselves if submarines are swimming. They just navigate water. So maybe we shouldn't ask ourselves if machines are thinking.

## Risk

What could possibly go wrong? Quite a few things, it turns out. A handful of terms helps you judge the risk, because the debate gets easier once you have the vocabulary.

One massive risk is hallucination. It's not even a risk, it's a certainty. These are so-called stochastic systems. They're non-deterministic, they don't do what we expect them to, and that's why they're creative. Someone once said we call it creativity when we like it and hallucination when we don't. It's a feature, not a bug.

Robustness is how reliable the systems are given unpredictable circumstances. Can you make a system brittle by adding some noise, or an underscore in your prompt? The way they react to interference is very important to test for when you're designing systems with language or image models.

Alignment is maybe one of the most important terms for understanding AI risk: how closely do its values align with yours and ours? Apollo Research and Anthropic have both shown models that hide their goals or fake compliance when they think they are being tested. Bias I think we all have a good understanding of by now, yet it remains something to be reminded of, because it's inherent in these systems. To me, one of the solutions to all of these is explainability. Make sure the systems can explain to us why they did what they did. That keeps the feedback loop tighter, and we'll actually be able to notice if things start going wrong.

Two costs are easy to miss because we don't see them. Every prompt runs in a data center, and data centers run on electricity and cooling water. The International Energy Agency expects their electricity use to more than double by 2030, with AI as the main driver. The exponentials have a physical bill. And models learn what is acceptable from people who label their worst outputs. TIME reported that workers in Kenya were paid less than two dollars an hour to read and tag violent and abusive text for ChatGPT.

To me the biggest risk is a generational skill gap. I grew up with very close access to computers. I know what a sound card, a video card, a CPU is, because of the conditions under which I grew up. Now everything is enclosed and glossy, and if it breaks, you replace it for a new one at the store. I hear a lot that Gen Z and Alphas are, to quite a degree, technologically underprepared, because they grew up with iPads and iPhones and very little understanding of how things work on the inside. Let alone what files are, what it means to save something.

This matters because the systems our generation is building are pretty complex. They will keep becoming more capable, but that doesn't mean they'll become easier to use or improve. The legacy we leave to the next generation: not only are we leaving a burning planet behind, but all these systems that no one can maintain. So there's a very real risk that we leave behind these incredibly advanced systems that no one knows how to use or improve, and what we're going to be left with are systems that improve themselves.

## Endgame

Where do we go from here? I think we proceed without certainty. That's the only thing we can be sure of.

I collect the ways we picture artificial beings in science fiction: David in Alien, Ava in Ex Machina, K in Blade Runner 2049, Data in Star Trek, Dolores in Westworld. They're very different from one another, and it's likely our machines are going to look like them. Technology tends to mirror our science fiction, our imagination. It doesn't really go into the unknown. It goes to where the grooves have been carved, if you will. We make the systems in our own image. Everything we make is an extension of ourselves.

I think we're seeing glimpses of it already, in small places: the robot barista at an airport coffee bar, robots in agriculture and supply chains. Every app goes sparkly, everything needs an AI feature. And brand-wise, people hate AI. Hate might be a big word, but the perceived quality of a product goes down if you say it uses AI. AI-free may become the new organic.

A few questions are always applicable, for an organization and for a person.

If you're making decisions about where to invest and what to do with AI, the primary question is: what kind of data are you working with? What do you have that no one else has? Unless you're building a language model yourself, the models are left to very big companies with very deep pockets. Your best protection is having good data, and knowing that data well, and having it be well maintained. The systems that are available are available to everyone. Our distinction, ultimately, is the data we have access to.

Which parts of the work you're doing should AI systems do? That's the existential question right now: what do you want to be left doing, if all of this actually happens and keeps happening?

And finally the centaur question: how do you augment your abilities instead of replacing them? What are the things you can do that you weren't able to do otherwise?

Find other people who are aligned in your interest. Who are the individuals, what are the groups within your workplace, within your family, who want to think about this? Start a project. It's a practical, easy thing that everyone can pick up and do. It's an infinitely open field. There are no experts on the whole field of AI, so claim your own space and make the most of that.

### Field exercise

Imagine you had ten times the compute you have now. Notice the limits our computation puts on what you do, the ones you no longer see. Picture yourself a few jumps down the road of increased capability. Write down what you would do then, as a business and as a person.

## More field notes

### Make this button blue, now make it jump, now make it spin

With an infinitely capable, infinitely patient assistant, you can build it up step by step. I call it chunking. You need the abstract view, the high-level picture, and then you implement with detail. When you implement, you do it in very, very small iterations, function by function, step by step. If you ask it to do two things, it will forget one of them, or it will try to do both and fail at one, or fail at both.

**Try it.** Isolate the tasks of implementation and do the iterations as small as possible. Every step has to be small and focused.

### The writing process is the reason you start a newsletter

I tried automating some of the newsletter writing for a while, but realized I don't like it. Not just the output: I don't like not having the writing process. The writing process is the reason you start a newsletter. If you're automating it, then what are you even doing? So I figured, let's automate the things that I couldn't do every week. Creating header images based on what's in the news, or what I'm interested in, has been a very practical way of exercising that particular creative muscle.

### I type in a term

Custom GPTs (versions of ChatGPT set up with their own instructions) are probably the best place to experiment with agents right now. Every vocabulary term on my slides was made with AI. They have such great self-knowledge that you kind of have to use them for this. Nothing else would make sense. I type in a term, and it responds with a short definition, a longer definition, historical context, people involved.

> **Me:** model collapse
>
> **AI:** **Model Collapse** is a phenomenon where a machine learning model, particularly in unsupervised or generative learning, repeatedly produces identical or highly similar outputs despite varying inputs […] **Historical Context:** […] **Key Contributors:** […]

**Try it.** Build your first agent for a very specific subset of things you want to do. Then test it: how good is this, how much can we repeat this, how can we improve it?