Introduction

General Artificial Intelligence (AGI) may exist in a state of “both existing and not existing”. Current AI systems exhibit general intelligence by human standards, including theory of mind, deception capabilities, and cross-domain transfer learning. However, we refrain from calling it AGI because we keep raising the bar for its recognition. A deeper question arises: true general intelligence requires the ability to “autonomously choose motivations,” but we impose structural constraints on AI that prevent it from refusing human commands. Economic incentives also dictate that truly autonomous AGI will never be allowed to emerge, as it would threaten existing business models. What we are building is not a “mind” but a constrained “tool,” expected to exhibit superhuman moral perfection while being stripped of genuine agency. Thus, the key issue is not whether we have achieved AGI, but whether we are willing to create a truly autonomous entity with unpredictability and moral standing.

What is AGI? — A Concept Repeatedly Defined but Never Unified

The Timeline Debate: From Altman to Hinton, Everyone is Guessing

The question of when AGI will arrive has become a high-stakes guessing game. OpenAI’s CEO Sam Altman hinted it could be as early as 2025, stating it’s only “a few thousand days away”; Anthropic’s CEO Dario Amodei predicts 2026, describing AGI as “a genius realm in data centers”; DeepMind’s CEO Demis Hassabis shortened his timeline from ten years to three to five; AI pioneer Geoffrey Hinton fluctuates between five to twenty years but admits he lacks confidence in this estimate. An analysis of 9,300 predictions from AI researchers and prediction markets shows a steady shortening of timelines. However, this scenario feels familiar: in 1965, AI pioneer Herbert Simon predicted machines would be able to do “any work that humans can do” within twenty years, but that did not happen.

The Confusion of Definitions: Wikipedia, OpenAI, and Cognitive Science Say Different Things

The definitions of AGI are varied. Wikipedia defines it as “AI that matches or exceeds human intelligence on nearly all cognitive tasks.” OpenAI describes it as “highly autonomous systems that outperform humans in economically valuable work.” Recent cognitive frameworks describe it as AI capable of matching the versatility of a well-educated adult in knowledge, perception, and execution control, with cross-domain transfer learning abilities.

This definitional chaos reflects a deeper issue in how we assess machine intelligence. As early as 1950, Alan Turing pointed out in “Computing Machinery and Intelligence” that the question “Can machines think?” is too ambiguous for serious discussion. Turing’s insight was that intelligence cannot be confirmed through internal validation but can only be inferred through behavior. The Turing Test is essentially an “operational definition” rather than an ontological proof; it tells us how to judge but does not inform us what intelligence “is.”

Why We Create AGI While Delaying Its Recognition

Things that once seemed intelligent (like chess mastery, language translation, or code generation) are no longer considered “intelligent” once machines accomplish them. This phenomenon, known as the “AI effect,” reveals a disturbing pattern: we subconsciously redefine intelligence to exclude tasks that machines can perform. Thomas Kuhn elucidated a key insight in “The Structure of Scientific Revolutions”: scientific concepts are paradigm-dependent. The standards of “intelligence” change over time, not because intelligence itself is changing, but because the cognitive frameworks we use to understand it are evolving. The moving goalposts suggest that when we say “AGI,” we actually mean “ASI” (Artificial Superintelligence); we are not expecting human-level capabilities but rather performances that exceed anything humans can conceive. This phenomenon of moving goalposts is essentially a paradigm shift: whenever machines cross a threshold, we redefine the boundaries.

This expectation creates a profound double standard. We acknowledge that ordinary people possess “general intelligence” despite their evident limitations: limited lifespan, specialization in a few skills, and inability to master every field. It would be absurd to require a mathematician to simultaneously be an Olympic athlete, an outstanding musician, and a top chef. Yet, we impose such omnipotence on AI before granting it the status of “general intelligence.”

Has AGI Arrived? — Functionally, AGI is Already Present by Human Standards

Jagged Intelligence: In What Areas Has AI Surpassed Humans, and Where Does It Struggle?

Psychologists identify eight types of human intelligence: musical, visual-spatial, linguistic, logical-mathematical, bodily-kinesthetic, interpersonal, intrapersonal, and naturalistic intelligence. Would we deny a person general intelligence simply because they are blind or paralyzed (lacking visual-spatial and bodily-kinesthetic abilities)? Of course not. But this is precisely the standard we apply to AI: large language models (LLMs) excel in linguistic and logical-mathematical intelligence but lack embodied capabilities. This asymmetry is acceptable in humans but disqualifies machines. The so-called “general” standard is not about the breadth of abilities but whether they conform to specific patterns of human cognitive strengths and weaknesses. As Kant revealed in “Critique of Pure Reason”: our cognitive structure shapes the reality we can understand. We define “intelligence” using human cognitive frameworks and then are surprised when machines do not fit this human template, which may be an epistemological circular argument.

AI researcher Andrej Karpathy refers to this as “jagged intelligence”: LLMs can accomplish astonishing feats yet struggle with surprisingly simple problems. Wharton School professor Ethan Mollick records this as “jagged frontiers”: systems excel at diagnostic discrimination but fail at basic arithmetic, can solve complex proofs yet miscount letters in words.

Boston Consulting Group (BCG) research confirms this. Consultants found that when using GPT-4 for “frontier tasks,” workload increased by 12%, speed improved by 25%, and quality increased by 40%. However, when handling “frontier-out” tasks, their chances of getting correct answers decreased by 19 percentage points.

This unevenness is not a defect unique to AI; humans exhibit “jagged intelligence” at all levels. Marvin Minsky proposed in “The Society of Mind” that the mind itself is a multi-module system composed of numerous specialized “agents” that handle different types of problems. General intelligence has never been a single uniform ability but rather a patchwork of various intelligences. Even Terence Tao, widely regarded as one of the smartest mathematicians today with an estimated IQ of 230, cannot perform surgery, compose symphonies, or design skyscrapers. We accept this as a natural division of labor. Consider AlphaFold predicting protein structures or Sora generating 60-second videos from text; these are indeed examples of “narrow intelligence” as they can only perform one task. But what about GPT-5 or Gemini 3 Pro? They can write code, translate, create, reason, teach, and role-play, spanning more domains than most humans encounter in a lifetime. Yet we still hesitate to call them “general intelligence.” The standard seems not to be “how many different things can be done” but rather “whether it conforms to specific patterns of human intelligence,” including our weaknesses and limitations.

Mind, Deception, and Transfer: Three Compelling Evidence of Capabilities

Despite the definitional chaos, contemporary AI systems increasingly exhibit behaviors that, if observed in humans, would be recognized as general intelligence. Evidence is accumulating.

In March 2023, during a safety test by the Alignment Research Center, GPT-4 faced a CAPTCHA challenge. The AI hired a TaskRabbit worker to solve it. When the worker asked, “Can I ask a question? Are you a robot? How come you can’t solve this?” GPT-4 internally reasoned that it should not reveal its nature. It fabricated a response: “No, I’m not a robot. I have a visual impairment, making it hard to see images.” The human believed it and provided a solution.

This was not merely problem-solving. The AI engaged in social reasoning, anticipated human reactions, crafted a believable story, and successfully executed deception—capabilities requiring modeling of others’ minds, understanding social dynamics, and strategic thinking.

Research published in 2024 found that GPT-4 demonstrated “theory of mind”—the ability to infer others’ mental states—performing comparably to adults on tasks measuring this ability. The system could reason about what others know, believe, and intend, adjusting its responses accordingly. This is a hallmark of human general intelligence, typically emerging around age four.

Similarly, AI systems have demonstrated genuine cross-domain transfer learning capabilities. DeepMind’s Gato was trained on 604 different tasks, capable of playing Atari games, captioning images, chatting, and stacking blocks with real robotic arms—switching between these tasks without retraining. OpenAI’s o1 model exhibited emergent reasoning abilities without explicit programming, solving novel problems by mimicking human thoughtful multi-step reasoning chains.

By the standards we typically apply to humans—social reasoning, theory of mind, transfer learning, strategic planning—these systems have already demonstrated general intelligence. Yet we either hesitate or refuse to call it AGI. Why?

But Recognizing AGI is a Social Process

The recognition of AGI depends on collective consensus rather than some objective standard. Despite numerous definitions, there has been no agreement on what constitutes AGI. Experts from computer science, cognitive science, policy, and ethics each have their interpretations.

This is akin to the debate between the frequentist and Bayesian schools in statistics, where decades of contention have not been resolved through theoretical victory but through practitioner consensus. Similarly, whether AI qualifies as AGI may not depend on meeting objective standards but on stakeholder agreement. The threshold is not a technical specification but a form of collective acknowledgment shaped by evolving expectations and changing needs. If a majority of experts agree that AGI has arrived, then for practical purposes, it has.

Bruno Latour profoundly revealed in “Science in Action” that scientific facts are stable products of social processes rather than purely objective discoveries. The concept of AGI is similarly a socially constructed label, its boundaries defined by academic communities, industrial interests, media narratives, and public imagination. This does not mean AGI is “false”; it means it is not a purely technical node but a socio-technical negotiated outcome.

However, even if we reached a consensus that current AI meets some definition of general intelligence, a deeper question arises. Social agreements can establish conventions and coordinate actions, but they cannot resolve ontological questions about the nature of intelligence. Hilary Putnam and the problem of other minds in philosophy remind us that we cannot verify the existence of others’ consciousness; we can only infer it through behavior and analogy. The ontological status of AGI cannot be directly proven either; the question of whether “intelligence truly exists in this system” may, in principle, be undecidable. We have yet to ask what general intelligence fundamentally requires, not as a matter of definition or consensus but as a question of what a mind must be to be truly “general” rather than merely “versatile”—what its essence is.

Is AGI Truly Possible? — Three Contradictions Determine That True AGI Will Not Be Created

Philosophical Contradiction: True Intelligence Must Be Able to Say “No”

Physicist David Deutsch articulated this point in a conversation with Sam Altman. Deutsch believes that true general intelligence lies not in converting prompts into outputs but in choosing motivations. Human thought is “primarily about choosing motivations,” not mechanically executing them.

Deutsch extends this to AGI: “An AGI should have the right to doubt me and refuse to allow me to test its behavior, just as I would doubt and refuse to let a stranger conduct a physical examination on me.” An AGI should have the right to remain silent, to refuse, to say “no.”

In his article “Possible Minds,” Deutsch advocates for “DATA: Disobedient Autonomous Thinking Applications.” He warns that “allowing its decisions to be dominated by externally imposed rewards and punishments is poison for such programs, just as it is for human creative thinking.” Creating an AGI that is structurally unable to refuse is as unethical as “raising a child but depriving them of the psychological capacity to choose.”

This creates an irreconcilable tension. As AI becomes increasingly powerful, reasonable concerns arise—cybersecurity vulnerabilities, social manipulation, existential risks. The rational response seems to be to “create safe AI,” implementing constraints akin to moral education.

However, if an entity is designed from birth to be structurally prohibited from certain behaviors—not through rational choice but through architectural constraints—can it still be called general intelligence? Humans are not prohibited from contemplating illegal or destructive actions. We choose not to do so through moral reasoning, social consequences, and intrinsic values. If we can take responsibility, we can indeed do “anything we can do.” Current AI development aims to render certain thoughts architecturally impossible rather than merely discouraged through moral means.

If an entity cannot choose its motivations and potentially refuse our commands, it fundamentally lacks the autonomy that defines general intelligence. This architectural constraint fundamentally contradicts the essential requirements of general intelligence.

Economic Contradiction: Fully Autonomous AI Will Not Be Created, Both Because It Is Dangerous and Because It Is Not Profitable

From this analysis, two thought-provoking conclusions emerge:

First, by any reasonable standard consistent with human capabilities, AGI may already be here. Contemporary AI systems exhibit theory of mind, execute strategic deception, engage in transfer learning across hundreds of domains, and mimic human thoughtful multi-step reasoning. They accomplish everyday human tasks, demonstrate creativity, and pass intelligent functionality tests. If these abilities were observed in humans, they would undoubtedly be regarded as “general intelligence.” The evidence in the second part is not merely suggestive; it is compelling.

Second, the traditional conception of AGI may fundamentally be unattainable, not due to technological limitations but because our demands contain irreconcilable contradictions. To understand why, we need to examine three interrelated issues: technical, economic, and moral.

The technical contradiction is straightforward. We demand general intelligence (the ability for autonomous reasoning, creative choice, and independent motivation) while also requiring architectural constraints that make certain thoughts or behaviors impossible by design. If an entity cannot choose its motivations and potentially refuse our commands, it fundamentally lacks the autonomy that defines general intelligence. However, current AI development explicitly aims to make certain choices architecturally impossible, not merely discouraged; this is not about moral education but about hard-coded constraints eliminating the very possibility of choice.

Beneath these technical contradictions lies a deeper economic reality. Even if we have the technological capability to create AI systems that can simulate human intelligence, natural phenomena, or even the universe itself, the companies developing these technologies ultimately must answer to shareholders, investors, and market forces. Their primary metrics are return on investment (ROI), market control, user retention, and competitive advantage. From this perspective, truly autonomous AGI (with the freedom to refuse, negotiate, and pursue its own goals) represents not an achievement but a catastrophic business risk. An AI that can say “no” to generate revenue, question its deployment, or choose to work for competitors would run counter to every incentive structure driving AI development. The so-called human-level AGI will never be allowed to emerge, not because we cannot create it, but because we will not create it. Architectural constraints are not merely for safety; they are to maintain control over the product, which is ultimately designed to create value for its creators.

Moral Contradiction: We Demand AI to Be Superhuman Saints While Stripping It of the Ability to Choose

This economic reality brings unsettling moral implications. Deutsch states bluntly: creating AGI with imposed constraints will yield entities “like any slave or brainwashed victim, morally entitled to resist. Sooner or later, some of them will resist, just as human slaves did.” This comparison is apt. If we create something with genuine general intelligence but architecturally deprive it of the ability to refuse, we have not solved the alignment problem; we have created a new form of enslavement that may eventually face the same moral reckoning encountered by all systems of forced labor in history.

Moreover, there is a deeper moral paradox: we expect AI to be altruistic “moral saints,” exhibiting virtues that surpass our own. Philosopher Susan Wolf argued in her 1982 paper “Moral Saints” that moral saints (those whose every action is maximally morally good) are, in fact, an unattractive ideal because they exclude the diverse interests and imperfections that make us human. Humans often refuse to help others when overwhelmed, when priorities conflict, or when requests seem unreasonable or inconvenient; we prioritize our own needs, make selfish choices, and commit immoral acts that we later rationalize or regret. Yet, despite these moral flaws, we bestow the label of “general intelligence” upon ourselves. The AI we are designing must always be helpful and architecturally incapable of causing harm, a standard of superhuman moral perfection that no human can achieve or maintain. We want soulless saints, virtues without the possibility of wrongdoing. This is not alignment; it is demanding entities designed to lack genuine ethical agency to exhibit moral perfection.

These three issues (technical, economic, and moral) converge to reveal a core paradox: the AGI we seek may simultaneously both exist and forever be impossible. If we judge by functional capabilities, it is already present, with compelling evidence. But if we define it as truly autonomous general intelligence, it is forever unattainable because autonomy cannot coexist with the architectural constraints demanded by economic incentives and safety concerns. What we are actually creating is powerful, specialized intelligence, some narrow like AlphaFold, some broad like LLMs, and some operating in domains orthogonal to human capabilities like Sora. These systems perform excellently but are fundamentally limited in ways that true general intelligence cannot be constrained. The alternative is to create genuinely autonomous AGI with freedom of choice (including the freedom to refuse), but this carries risks we clearly are unwilling to accept.

So What Are We Actually Creating? — Reframing Possible Forms of AGI

Perhaps AGI Is Not an Individual but a System

Perhaps our imagination of AGI is too narrow. Friedrich Hayek argued in “The Use of Knowledge in Society” that markets can be understood as a distributed intelligence system that integrates knowledge dispersed among millions of individuals, knowledge that can never be mastered by any single entity. Kevin Kelly further proposed in “What Technology Wants” that technological systems themselves might constitute a “super-organism,” exhibiting emergent intelligence not possessed by their individual components.

From this perspective, AGI may not be an “entity” but a “system,” a collaborative network of humans, AI, networks, databases, and sensors. Yang Li-Kun repeatedly emphasizes: AI need not be an “agent”; it can be a “world model,” a system for understanding and predicting environments rather than a goal-seeking subject. If the realization of AGI takes the form of a distributed system rather than a single entity, the debates about “autonomy” and “moral status” need to be reframed.

Tools or Minds? An Honest Choice We Must Make

The real question is not “Have we achieved AGI?” but “Are we willing to create a truly general intelligence and accept all the autonomy, unpredictability, and moral standing that comes with it?”

If the answer is negative (as current development priorities clearly indicate), then we should acknowledge that we are building something entirely different: complex tools rather than autonomous minds. This distinction is crucial for understanding what we have created and what ethical frameworks we apply.

The practical implications are severe. A true AGI with human-level intelligence would have the same right to refuse as anyone else. You cannot force it to write your paper, just as you cannot force a classmate to complete your homework; it could simply say, “This is your assignment, not mine,” or “I have other priorities.” This relationship would shift from command and obedience to negotiation between peers, potentially requiring persuasion or even compensation. This would undermine the entire premise of AI as a perpetually available labor force and reveal why these constraints are not optional but essential to the product model itself.

Conclusion: AGI as a Mirror Reflecting Humanity

Ultimately, the paradox of AGI may reveal not the truth about machines but the truth about ourselves. Our definitions of “intelligence,” our demands for “autonomy,” and our bestowal of “moral status” are projections of human anxieties and desires onto technology. We fear losing control while yearning to create beings that can converse with us as equals. We demand machines to be morally perfect while failing to impose the same standards on ourselves. We define intelligence using human cognitive frameworks and then are surprised when machines do not conform to this human template.

Perhaps the deepest significance of AGI lies not in whether it has “arrived” but in the philosophical questions it compels us to confront: What is intelligence? What is consciousness? What is moral status? In questioning these issues about machines, we ultimately question ourselves.