My rationale behind this cartoon may be a bit difficult to explain, but I’ll try.
In essence, I believe that the concept of a true AGI — an artificial general intelligence comparable to a “god-like” AI — is logically impossible. Don’t get me wrong: AI systems will continue to improve dramatically. They are already unmatched as repositories of factual knowledge (and, frankly, getting a definitive answer from a human can be just as unreliable as an AI “hallucination”). They are also becoming increasingly adept at logical reasoning. I would even venture to say that perhaps humans are not as special as we like to think — and that AIs might, one day, attain something akin to sentience.
However, decision-making — even when grounded in true facts and valid logic — inevitably depends on _context_ and the _individual judgment_ applied to numerous competing factors. Think back to mathematics: you can assert that 2 > 1, but there’s no universal way to say that the vector (1,2) is “better” than (2,1). You can define a metric _m_ such that _m_(1,2) > _m_(2,1), but for real-world problems — where the components might represent knowledge, beliefs, or values — that reduction in dimension will differ for every individual. The same will hold true for each AI, unless they are all trained identically, use the same methods, share the same biases, and produce deterministic outputs.
Humans attempt to mitigate this challenge through collective decision-making — what we call democracy. In essence, we take an average of the beliefs and judgments within a group and act on that consensus. This works reasonably well, as long as no single group becomes overwhelmingly dominant and distorts the balance — much like the imbalance depicted in the cartoon.
A single AGI, no matter how advanced, would face the same limitation as a single human: it could operate on vastly more information, but it could never make a decision that is universally good for everyone. It might optimize for outcomes that benefit the majority, yet individual well-being would still be reduced to a mere pixel in a larger picture.
In conclusion, it is the notion of ‘generality’ in AGI that I believe will remain forever out of reach.
Have fun,
Rüdiger
