I've been working on AI/AGI/ASI or what we used to call "Human Level Intelligence" since ~1997 when Unreal Engine came out. I have a clear visual memory of being 13 and standing in our apartment in Webster TX, telling my mother that I want to train a NPC to be smarter than the combined knoweldge of humanity. One of my biggest complaints back then as a teenager was that literally nobody had a satisfying answer on "How will we know when we got there?" Later, in preparation for my Masters thesis, the best answer I found was around Hernandez-Orallo's 2010 anytime intelligence test.
Unfortunately the egoist, anthropocentric, narcissists that run the human species, can only fathom ourselves as the ultimate being. This is why almost every god is effectively just a scaled version of a human, you can discover this yourself from every prophetic work written as I compiled already. Hence why the entire discourse around “AI alignment” keeps taking this incoherent, multi-layered, historically contingent bundle of heuristics, myths, and post-hoc rationalizations and treating it as some kind of empirical gold standard for what intelligent systems should emulate.
This is deeply confused. The idea that human behavior should set the boundary conditions for advanced artificial systems collapses immediately once you recognize how little of human reasoning is actually grounded in the evidence-seeking structure science demands. The replication crisis in social sciences alone demonstrates that humans fail even when actively trying our best to be rigorous.
Humans don’t behave according to stable epistemic rules; they behave according to social pressures, inherited cultural myths, metabolic constraints, and local incentives. We know from behavioral economics that the majority of “rational” human behavior is anything but predictable or rational. We know from anthropology that "core" moral intuitions vary wildly depending on ecological niche and historical accident. We know from neuroscience that most cognitive operations are not explicit reasoning but predictive filtering and error-correction mechanisms tuned for survival rather than truth. Anyone using human behavior as a normative vector for training intelligent systems is implicitly saying “AI should emulate an epistemologically unstable system that barely tracks reality in the first place.”
The only predictable process we’ve ever had for progress (defined as increasingly fine grained and longer horizon accuracy of prediction of a defined and measurable system) in any domain is the scientific ideal: A process that explicitly rejects subjective intuition in favor of measurement, model revision, and mechanical tests against the "ground truth" of the measured world. If artificial systems are going to model the world, the only successful epistemology is a mechanical loop of measurement, model, intervention, and correction. Anything short of this will encode bullshit human myths designed to maintain some existing or transitioning biological power structure, which I wrote about in 2022 with my closing paragraph in Myth of Scarcity:
The question of “Alignment” then is an Absurd question - as humanity cannot align internally due to our structural impediments. To anyone who is worried about “Aligning” AI globally, your charge is this: Invert structural incentives in your organization to demonstrate cooperation rather than competition, and sharing rather than hoarding as much as possible. This is the only way to create data that is de facto embedded with cooperation, sharing and trust. If we stay on the current path, we will continue to generate data teaching our systems how to destroy each other rather than care for each other. This is an existential threat that cannot be buttressed by any number of laws or rules enforced by hierarchies.
This unflinching and devastating indictment of human perception was what drove Francis Bacon to first sketch out the Novum Organum (which you haven't read) and what cybernetics later formalized as feedback-driven control (which you probably haven't heard of). The scientific ideal is not a set of conclusions; it’s simply the epistemic process of eliminating possible solutions based on measurable hypotheses and tests. More importantly it’s the only process that reliably produces higher precision prediction of larger systems over time. If you want AI to converge toward SOMETHING, converging on an epistemic discipline that already outperforms human intuition by orders of magnitude is the only answer. Precision manufacturing, particle physics, climate modeling, genomics, or any domain where human belief has been replaced by instrumentation, again I already proved that this is how humans operate in Faith is invisible
I measure AI progress against that: the capacity to sense the world, generate mechanistic predictions, test them, and compress the resulting knowledge into models that outperform the biological approximations. Humans are useful data points only because they exist as actors that can be instrumented, not because they are coherent. The benchmark cannot be emulating how people behave, but rather the predictive models produced by the scientific process. That is the only human invention with an unbroken record of increasing predictive power across centuries.
The arrival of machine intelligence is finally forcing the necessary realization that human behavior is not the north star for anything. It’s a local minima with a screeching noise floor from which we’ve been trying to climb for Millions of years. Machines don’t need to align to us; they need to align to the only thing we’ve ever built that actually aligns to the world: Scientific rigor
Copyright (c) 2025 Andrew Kemendo