Laplace's Demon

When I was around 17 years old I came to the conclusion that the most important thing humanity could possibly do was build Laplace's demon. I've more or less spent the rest of my life working on that problem.

Laplace's demon is a thought experiment proposed by Pierre-Simon Laplace in the early nineteenth century. Imagine an intelligence that knows the complete state of the universe at some instant and understands the laws governing everything in it. If the universe is deterministic, that intelligence could calculate everything that came before and everything that comes afterward. The entire universe becomes, at least in principle, observable from a single state.

This is obviously a strange thing to organize a life around, I can’t tell you why this became my goal, and for most of my life I haven't had much reason to spend time trying to explain it because there wasn't really anywhere useful to start. Artificial intelligence becoming part of popular society has made the conversation somewhat easier because people are now encountering, for the first time, systems that can do things we previously assumed required human intelligence. That gets us at least a few steps closer to the actual thing I've been interested in.

There are plenty of reasons from modern physics and mathematics to think the literal version of this is impossible. Quantum mechanics presents obvious problems. Computation itself may impose limits, and any demon inside the universe has the additional problem of modeling itself which I address to a degree within my GTC framework and future work I want to put out related to these problems.

I don't find any of these objections particularly interesting as reasons to abandon the project, because I don't think humans are remotely capable of knowing which of the things we currently consider fundamental limitations are actually fundamental limitations.

A dog cannot understand f=ma, but this tells us something about dogs rather than something about Newtonian physics. Humans are certainly more capable than dogs, but I don't know of any reason to believe that evolution happened to produce a primate whose cognitive abilities correspond neatly with the limits of knowable reality. It seems much more likely that there are enormous portions of reality that are as inaccessible to us as notational physics is to a dog.

So I have never really thought my job was to figure out how to build Laplace's demon. My job has been to build something capable of building something better than itself, which can then build something better than itself, and to continue that process until eventually the things doing the work are capable of answering questions that humans cannot.

That is basically why I got into artificial intelligence.

Prediction

The common thread through most of my work has been prediction, and humans have a peculiar relationship with the future because we are extremely good at predicting some things and unbelievably bad at predicting others. I can throw a ball at another person and their brain will continuously model its trajectory like discussed in my stream forecasting hypothesis, move their body and place their hand in roughly the correct location without either of us knowing the equations involved. We can build a bridge with a reasonably good expectation that it will still be there tomorrow (see my discussion of faith in empirical systems).

As soon as the number of variables becomes sufficiently large, however, our ability to do this collapses. This isn't necessarily because the people involved are stupid, though it’s all relative, there is simply more happening than a human being can model.

I've always found the standard response to this strangely unsatisfying. We accept that decisions have unknowable consequences, develop various heuristics for making better decisions anyway, and then treat judgment under uncertainty as an effectively permanent condition of existence.

My instinct has always been the opposite: make the future increasingly predictable at the most granular level.

If a decision has ten possible actions, I want to know what happens after each one. If each of those produces another hundred possible states, I want those too. If the relevant consequence doesn't appear for thirty years, then I want to model all thirty years.

Building the thing that builds the thing

A sufficiently general system should be able to observe some part of reality, construct a model, use the model to generate possible future states, compare those predictions against what actually happens, and modify the model accordingly. There is nothing particularly revolutionary about that idea, it’s how proper science already works.

The important transition happens when the system becomes better at improving this process than the people who created it. I don't know what the systems several generations down that sequence look like. In fact, not knowing is the entire point. If I could specify them in detail, they wouldn't represent the kind of improvement I'm interested in.

I don't particularly need GTC to be the final correct description of reality. I would be extremely surprised if something I developed with a human brain turned out to be the final description of anything. What I care about is whether it helps bootstrap a system capable of producing a better model than the one I gave it. If something descended from GTC eventually proves that GTC is wrong, but could not have existed without it, then GTC worked.

The horizon

If you keep following this process conceptually, you eventually arrive back at Laplace's demon.

Each generation makes some previously inaccessible portion of reality accessible. Better models enable better experiments. Better experiments produce better measurements. Better measurements reveal relationships that allow better models. Those models may produce entirely new physics, which enables different computation and different instruments, which expose still more of reality. So Laplace's demon functions for me as a horizon rather than an engineering specification. The direction is toward a universe in which progressively more of reality, past, present and future is accessible to an intelligence process.

At some sufficiently extreme limit, which is that horizon, I'm no longer sure the distinction between the model and the universe means anything. If such a state is physically possible, I don't know what happens when it is reached. That's why it's a horizon rather than something I can describe.

Why this matters now

For most of my life there was an enormous distance between this cosmology and anything happening in ordinary life. We are now building non-human systems that can outperform individual humans at particular forms of modeling, prediction and reasoning. This is still extraordinarily far from the thing I'm describing, but it introduces a relationship that humanity has never had to confront at scale: humans may not remain the most capable entities available for every form of intelligent action.

That has implications that are simultaneously mundane and profound.

The mundane version is that somebody may lose a job because a machine performs the function more cheaply. That matters enormously to the person who loses the job, particularly because we have built societies in which access to resources is strongly connected to economic usefulness.

The bigger issues is that there is no obvious reason the process stops at jobs.

There is nothing cosmologically important about humans writing software, driving trucks, performing scientific research, managing companies or even designing artificial intelligence systems. These are things humans do because, until recently, humans were the systems available to do them. If other systems become more capable, then human desires increasingly cease to determine what happens merely by virtue of humans being the most capable actors.

I think people correctly recognize something frightening in this.

The problem with other people

The world people live in is not, in any meaningful sense, the world each of them chose.

It is mostly a world produced by the people and institutions with enough energy, resources, attention and organizational capacity to shape it. Everyone else adapts to the resulting conditions according to their own circumstances. There’s a quote from Thucydides: “the strong do what they can and the weak suffer what they must.”

Technology amplifies this enormously. A comparatively tiny number of people can now make decisions that change the lives of billions of people they will never meet.

Strong Artificial Intelligence or Artificial General Intelligence increases that asymmetry again.

If the trajectory I have described continues, then it ultimately crowds out enormous categories of human action. There is no amount of prediction that makes different desires become the same desire. So I'm left with the same problem everyone else is, just at a somewhat ridiculous scale.

I have to act without knowing whether the future I am helping produce is the future other people would choose. Doing nothing doesn't resolve this. The world continues without my participation and the people with different objectives continue acting. Abstention is simply another way of selecting which forces shape the future. There is no outside.

Where that leaves me

I want to make reality increasingly observable. I want to build systems capable of producing better systems. I want those systems to discover things that humans cannot discover and then use those discoveries to make still more of reality accessible.

I also think the transition toward systems more capable than humans is extraordinarily dangerous, particularly because it is occurring inside human institutions already characterized by enormous differences in power.

If increasingly capable intelligence is going to expand the causal reach of whoever controls it, then understanding what is happening and making that knowledge available outside the institutions building the technology becomes increasingly important. People cannot participate meaningfully in something they cannot observe, although making something observable obviously doesn't guarantee that they will participate.

Reduce the amount of reality that is inaccessible. Build systems capable of improving that process. Let those systems build their successors. Keep going until reality tells us we cannot.