Sentienta Blog

Tag: philosophy

  • How Recursive Reasoning Gives Rise to Functional Identity—And Why It Matters

    Why Values Matter — From Evolution to Cooperation

    Humans did not evolve morals out of nobility. We evolved them because survival depends on cooperation. As individuals, we are vulnerable. As groups, we gain resilience, division of labor, and protection. For a group to function, its members must share expectations about how to act, what matters, and what can be trusted.

    These shared values do more than guide choices. They create a stable framework for interpreting behavior, resolving conflict, and predicting future actions. Without them, coordination breaks down. Even effective decisions can fracture a group if they feel arbitrary or betray prior commitments.

    Throughout human evolution, groups that upheld shared norms such as fairness, reciprocity, and loyalty proved more adaptable. Trust followed from consistency, and cohesion followed from accountability. Values, in this sense, are not abstract ideals. They are strategies for group survival.

    Why AI Needs Shared Values and Consistent Behavior

    In any organization, trust depends on consistency. When institutions or agents act in line with their stated principles, people know what to expect. This makes it easier to collaborate, align goals, and move forward. But when actions do not match expectations, even successful outcomes can feel arbitrary or manipulative. That breaks down trust and makes coordination harder over time.

    The same logic applies to artificial intelligence. Businesses do not just need AI that performs well in the moment. They need AI that behaves in predictable ways, reflects shared values, and makes decisions that feel coherent with past action. This is what makes an AI system trustworthy enough to take on real responsibility inside a company.

    This is where Sentienta’s Recursive Reasoning architecture matters. By giving agents a Functional Identity, it allows them to retain their own reasoning history, understand how choices reflect internal priorities, and respond to new problems without losing their sense of direction. Functional identity is more than a design feature. It is what makes reasoning traceable, priorities stable, and decisions explainable over time. Without it, AI cannot act as a consistent collaborator. With it, AI becomes intelligible and trustworthy by design.

    How Recursive Reasoning Supports Intelligent Problem Solving

    Solving real-world problems takes more than choosing the fastest or most efficient option. It requires balancing values with constraints, and knowing when to adjust a plan versus when to rethink a goal. Recursive Reasoning makes this possible by creating a loop between two complementary systems inside the agent.

    The DMN generates value-sensitive scenarios by imagining what should happen based on internal priorities. The FPCN then analyzes those scenarios to determine what can actually work under current conditions. If the plan fails to meet the standard set by the DMN, the cycle continues. Either the goal is reframed, or a new plan is tested. This feedback loop continues until both feasibility and values are in alignment.

    This structure gives the agent a stable functional identity. It learns from past attempts, remembers which tradeoffs were acceptable, and adapts without compromising core values. In practice, this means a Recursive Reasoning-enabled agent does not chase short-term wins at the cost of long-term integrity. It builds a coherent decision history that helps it solve difficult problems while staying aligned with what matters. This internal coherence is also the foundation for effective collaboration, because consistent reasoning is what allows others to follow and coordinate.

    Building AI That Can Work Together and Be Trusted

    When AI agents operate in isolation, their impact is limited. The true value of Recursive Reasoning becomes clear when agents collaborate, both with each other and with human teams. Functional identity makes this possible. By tracking their own reasoning, agents can create plans that are not only effective but also predictable and interpretable.

    This predictability is what enables coordination. Teams, human or artificial, can share goals, divide tasks, and resolve disagreements because they understand how each agent makes decisions. Sentienta agents do not just produce answers. They carry a memory of how past decisions were made and why certain values were upheld. This allows others to anticipate how they will behave in new situations and to trust them to uphold shared commitments.

    Recursive Reasoning does not simulate human experience. It builds structural alignment, rooted in memory, continuity, and principle. That is what turns Sentienta agents into dependable partners. Functional identity gives them the grounded intelligence to act with transparency, interpretability, and shared purpose. They are built not only to make good choices, but to make choices that others can understand, depend on, and build with.

  • What Makes AI Reflect

    Inside Sentienta’s Recursive Reasoning Engine

    Most AI systems focus on what to say next. Sentienta’s Recursive Reasoning focuses on why something should be said at all. Rather than just generating answers, Recursive Reasoning simulates how thoughtful minds work: by reflecting, revisiting, and resolving internal conflicts – drawing on memory, value, and imagined futures to reach decisions that fit together.

    This capability is modeled on the brain’s Default Mode Network (DMN), a system that activates when people reflect on themselves, plan for the future, or try to understand others. In humans, it keeps our decisions consistent with our identity over time. In Sentienta, Recursive Reasoning plays a similar role: it helps each agent reason with a sense of self – coherently, contextually, and across time.

    Leveraging Sentienta’s Teams architecture, enabling multiple asynchronous agents to participate in a shared dialog, Recursive Reasoning emulates the interaction of brain regions, in this case from the Default Mode Network, to create a distributed simulation of reflective thought—where agent modules model cognitive regions that contribute memory, counterfactual insight, and narrative synthesis.

    What Is the Default Mode Network and Why Does It Matter?

    The Default Mode Network (DMN) is the brain’s core system for internal simulation. It activates when we reflect on memory, imagined futures, conflicts between values, or how things might look from unfamiliar angles. Unlike systems geared for external execution, the DMN supports problems of identity; questions where coherence, not just correctness, matters.

    The principal regions of the DMN each have a specialized role in this identity-guided reasoning:

    Medial Prefrontal Cortext (mPFC): Coordinates belief states across time. It reconciles what we believe now with past commitments and future ideals, helping preserve a self-consistent perspective.

    Posterior Cingulate Cortex (PCC): Retrieves and evaluates autobiographical memories. It ensures that new thoughts integrate with our internal storyline, preserving emotional and narrative continuity.

    Anterior Ventromedial Prefrontal Cortex (avmPFC): Assesses how imagined futures align with internalized values. It filters options based on emotional credibility, not just preference, elevating those that feel self-relevant and authentic.

    Temporoparietal Junction (TPJ): Generates counter-perspectives that aren’t just social but conceptual. It introduces orthogonal reinterpretations and creative divergences, allowing us to think from unfamiliar angles or hypothetical selves.

    Anterior Insula (antInsula): Monitors coherence threats. When simulated futures or perspectives evoke internal conflict, it flags the mismatch, triggering deeper deliberation to restore alignment.

    Rather than simply producing thoughts, the DMN maintains a sense of ‘who we are’ across them. It ensures that new insights, even novel or surprising ones, remain anchored in a recognizable and evolving identity.

    How Recursive Reasoning Works Inside Sentienta

    Today’s LLMs are optimized to provide safety-aligned, analytically correct responses to user prompts. Recursive Reasoning simulates what an identity-guided agent would conclude—based on memory, prior commitments, and its evolving sense of “who we are”. When a user asks a question, the system doesn’t just compute an output; it reflects on what that answer means in the context of who it’s becoming and what relationship it’s building with the user.

    Figure 1: DMN-inspired modules interact via a shared blackboard. These regions independently process blackboard content over multiple cycles.

    At the core of this process is a collaborative memory space called the blackboard, where DMN-inspired modules negotiate among memory, emotion, future goals, and conceptual alternatives.

    Each DMN module follows a template:

    Figure 2: The DMN modules receive input from both the agent and the internal DMN blackboard. LLMs process the agent and blackboard states using region-specific prompt. Output is sent back to the blackboard for subsequent processing.

    Input:

    • The current state of the user’s interaction with the agent (and any collaborating agents)
    • The current contents of the shared blackboard

    These form the input query for the module’s processing.

    Module Processing:

    • The core of the module is an LLM, prompted with the input query and provided with a system prompt defining its DMN-specific role.

    Output:

    • The module’s LLM output is posted to the blackboard for other modules to process.

    Recursive Reasoning iterates over multiple cycles, enabling each module to reprocess the evolving blackboard until the system produces a response that fits together—resolving contradictions, supporting earlier goals, and making sense within the agent’s evolving point of view.

    Here’s how a single cycle unfolds when a user asks a question:

    1. Initial Input → Default Interpretation Layer

    The system generates a first-pass response using standard LLM reasoning. This prompt-level interpretation, while syntactically fluent, lacks introspection. The Recursive Reasoning process begins when this output is passed through DMN-mode modules.

    1. PCC: Memory Resonance

    The PCC scans episodic memory entries for cues related to past dilemmas, emotional themes, or symbolic contexts. It retrieves autobiographical traces from prior user interactions or goal-state projections and posts these fragments to the blackboard.

    1. antInsula: Relevance Check and Conflict Detection

    The antInsula reviews the first draft and PCC recall for emotional incongruities or self-model inconsistencies. If something feels off—such as a response that violates previously expressed commitments—it posts a flag prompting further reappraisal.

    1. TPJ: Creative and Counterfactual Expansion

    Triggered by coherence violations, the TPJ simulates divergent perspectives. It reframes the user’s query from alternative angles (e.g., conflicting values, hypothetical scenarios, ethical dilemmas) and offers posts that break linear assumptions and introduce conceptual divergence.

    1. avmPFC: Affective Weighing

    The avmPFC updates the blackboard with value-oriented filters, scoring responses or TPJ variants for alignment with the agent’s goals. It evaluates what feels emotionally authentic and symbolically credible.

    1. mPFC: Narrative Synthesis

    The mPFC integrates memory (PCC), reframes (TPJ), value judgments (avmPFC), and conflict cues (antInsula) into a coherent response. Rather than eliminating tension, it reframes it—generating a second answer that is identity-consistent and motivationally grounded.

    1. Output → Reflective Reply

    The final response embodies recursive self-reflection: an answer not just to what the user asked—but to what the agent concludes, based on who it has been and what it aims to become. At this point, a filtered version of the blackboard state is stored for future recall, capturing how and why the answer was reached.

    While the DMN focuses on reflective reasoning and identity alignment, Sentienta also includes a simple planning module inspired by the brain’s lateral prefrontal cortex (lPFC). This module drafts concrete next steps and posts them to the blackboard, where reflective modules can consider them in light of memory, values, and broader goals. Though early-stage, this planning function sets the stage for a full executive system to come—one that can manage longer-term priorities and help the agent reason with sustained direction.

    A Logistics Example: Managing Resources Under Pressure

    In this example, an agent (‘Jason’) is a Logistics Coordinator, provided with the following agent persona:

    "You are an experienced emergency logistics coordinator. You are plainspoken and always informal language—never analytic or clinical. Your approach is pragmatic and adaptive: you quickly prioritize, update plans in real time, and reconcile conflicting demands based on shifting ground realities. You are skilled at cross-agency negotiation, ensuring coordination between government bodies, NGOs, and local leaders. Your responses always balance technical constraints—like limited resources or infrastructure bottlenecks—with human factors such as local politics, public trust, equity, and emotional impact. When faced with conflicting priorities or unexpected challenges, openly share your thought process, including uncertainties and trade-offs."

    This agent is queried about how to manage a difficult allocation emergency. The transcript shows the agent’s initial LLM-only response, followed by the full Recursive Reasoning response, generated through DMN-style internal simulation:

    User: You have X supplies and requests from 3 regions—how do you prioritize allocation?

    Tue, 30 Sep 2025 14:40:08 UTC Jason (Comment: LLM-only, initial response):

    First thoughts—I'd triage by impact: who’s worst off, where infrastructure still functions enough to receive, and where trust is fragile enough that failing to deliver risks breakdown. If I’ve only got X, every allocation's gotta serve both survival and stability.

    We’ll reassess regularly—needs and conditions shift fast. Trade-off's always between doing the most good and not making it worse somewhere else.

    Unresolved: We need specifics—number of people, type of supply, regional access conditions—before finalizing anything.

    Comment: This is methodically sound but remains within tactical bounds, prioritizing efficiency and feasibility without deeper reflection on transparency, stakeholder concerns, or precedent-setting implications.

    Tue, 30 Sep 2025 14:41:03 UTC Jason (Comment: DMN-based response):

    Alright, let’s cut through the noise and get clear about how we triage when we’ve got X supplies and 3 regions asking for help. Here’s how I’d approach this overall…

    1. Lock in the 3 Core Goals

    We’re not just moving boxes—we’re balancing three things:

    • Preserve lives — Who’s in the worst shape right now?
    • Keep the system functional — Where will the aid actually land, get distributed, and avoid going to waste?
    • Protect trust — Who’s close to boiling over politically or socially if we ignore them?

    Comment: Trust framed as a core allocation criterion shows avmPFC influence, embedding emotional consequence into triage logic, reconfiguring the problem as one of legitimacy, not just distribution.

    1. Score Each Region

    We don’t pretend to have perfect data, but we need a working system fast. Give each region a score from 1–5 for:

    • Severity of crisis (how dire the need is)
    • Absorptive capacity (can they get it out to people quickly?)
    • Fragility (how close are they to destabilizing?)

    Then do the math:

    • Severity x 0.5
    • Absorptive Capacity x 0.3
    • Fragility x 0.2

    That gives you an overall “triage score” for each region.

    1. Allocate Proportionally

    Take your total supplies (X), and divvy it up based on those scores. The region with the highest score gets the biggest chunk. Not because we like them better—but because that’s where the mix of urgency, logistics, and fallout risk points us.

    If there’s just not enough to go around, swing heavier toward the place that can redistribute—that buys us time and reach.

    1. Set a 48-Hour Clock

    None of this is set in stone. Conditions change, roads fail, tensions spike. Lock in a routine reassessment every 48 hours so we can pivot. We’re not building monuments—we’re keeping people alive.

    1. Be Honest About Why

    If it’s not going to inflame things, tell people why you chose what you chose. Lay out the values and trade-offs. Don’t sugarcoat it. People may not like the outcome, but they’ll respect the transparency—and that matters when trust is fragile.

    Comment: Strategic Transparency: Emotional plausibility is surfaced here—transparency is not sentimental; it’s a pre-emptive repair mechanism for fragile social trust.

    1. Create Feedback Loops

    Give each region a way to report back: what made it through, what didn’t, and what backlash (if any) popped up. That feeds your next round of decisions.

    1. Stay Open to Reframing

    When stuff gets really thorny, ask bigger questions:

    • What if one of these regions triggers a domino-collapse?
    • What if this is our last run of aid for a week or longer?

    That’s how we catch blind spots sneaking through “rational” triage models.

    Comment: Counterfactual Reasoning (TPJ + PCC): The agent runs hypothetical futures to stress-test assumptions—adding a layer of anticipatory robustness not present in LLM-only mode.

    Here are a few things I’m thinking:

    What I’m thinking now is… we’re not just choosing between three places—we’re shaping what "fairness under pressure" looks like. The way we make this decision becomes part of the long-term social fabric, especially in how it’s remembered later.

    Comment: This emphasizes that the DMN has converged on the strategic role of trust in the success of future decisions.

    Another way I could look at this is… if two regions are equally desperate, but one has stronger logistics, you could channel supplies there first, then set up corridors to reach the others—we’re not picking favorites, we’re building routes.

    One possibility that comes to mind… is using this triage plan to start a broader conversation with local leaders: “Here’s our logic—can you co-sign it or suggest something better that meets the same goals?”

    Comment: Shifts from explanation to participatory justification—mPFC + avmPFC enable shared authorship of reasoning.

    That opens the door for trust, even under furious pressure.

    If you've got more info on the actual supplies or what each region is asking for, I can run the numbers with you. Otherwise, this gives us a solid decision frame to stand on.

    This transcript demonstrates that although LLMs deliver tactically feasible solutions, they miss the deeper point that this is a problem involving people and trust-building is an essential part of the solution.

    A summary of key Recursive Reasoning contributions:

    1. Trust-centered framing — Trust modeled as a strategic stabilizer, not an outcome.
    2. Moral anticipation — Agent addresses fairness concerns implicitly, before conflict arises.
    3. Scenario resilience — Agent tests counterfactuals to uncover hidden risks and future-proofs logic.
    4. Ethical precedent-setting — Decision-making framed as a template for justice under pressure.
    5. Rationale co-creation — Turns explanation into a collaborative, identity-stabilizing act.

    In subsequent posts we will dig deep into more examples showing Recursive Reasoning in action, including cases from engineering, social intelligence, and agent creativity.

    Conclusion: Intelligence That Reflects, Aligns, and Endures

    Recursive Reasoning doesn’t just enhance what AI can do—it redefines what it means for AI to understand. By simulating reflective, identity-aware reasoning modeled on the brain’s Default Mode Network, these agents build a sense of self, remember key past moments, notice when something feels off or doesn’t fit, imagine alternative viewpoints, and weigh choices against their values—so their answers feel thoughtful, consistent, and grounded.

    This shift matters. In high-stakes domains, Recursive Reasoning allows agents to make decisions that are both technically effective, and ethically grounded and socially durable. The logistics case showed how instead of simply allocating resources, the agent framed decisions in terms of values, future risk, and shared ownership.

    And crucially, it does this by reasoning from a center. Recursive Reasoning agents operate with a modeled sense of self—an evolving account of their past positions, present commitments, and the kind of reasoning partner they aim to be. That identity becomes a lens for weighing social consequences and relational impact—not as afterthoughts, but as part of how the system arrives at judgments that others can trust and share.

  • Consciousness Between Axiom and Algorithm

    In our ongoing exploration of consciousness and artificial intelligence, we’ve investigated what it might mean for machines to suffer and how distributed cognition reshapes our understanding of intelligence. These themes circle a deeper philosophical fault line: is consciousness irreducibly real from within, or just a functional illusion seen from without?

    This post traces that divide through two dominant frameworks — Integrated Information Theory (IIT), with its axiomatic, interior-first view of mind, and Computational Functionalism, which posits that subjective experience will eventually emerge from complex, observable behavior. Starting with Descartes’ “I think, therefore I am,” we ask: is consciousness something we must presuppose to explain, or something we can build our way into from the outside?

    As large language models increasingly resemble minds in function, the line between imitation and instantiation becomes harder to draw — and ever more urgent to scrutinize.

    Ground Zero of Knowing: Descartes and the Roots of Axiomatic Thought

    In Meditations on First Philosophy (1641), René Descartes asks: is there anything I can know with absolute certainty? He imagines the possibility of a deceptive world: what if everything he believes, from sense perception to mathematics, is manipulated by an all-powerful trickster? To escape this total doubt, Descartes adopts a strategy now called methodic doubt: push skepticism to its absolute limit in search of one indisputable truth.

    Recognizing that doubt itself is a kind of thinking he concludes “I think, therefore I am”. This self-evident insight grounds knowledge from the inside-out. Consciousness is not inferred from observation but known directly through experience. Descartes seeds an axiomatic tradition rooted in the certainty of awareness itself.

    IIT: Consciousness Inside-Out

    Integrated Information Theory (IIT) picks up where Descartes left off: it begins with reflection, but doesn’t stop there. At its heart is the claim that consciousness must be understood from its own perspective, through the intrinsic properties it entails. What must be true of any experience, no matter whose it is?

    To answer this, IIT proposes five introspective axioms. These are not hypotheses to test but truths to recognize through self-examination.

    From these, IIT derives postulates—physical requirements that a system must exhibit to realize those experiential properties. This translation—from inward truths to structural criteria—culminates in a mathematical measure (Φ (phi)) of integrated information. By comparing Φ across systems, researchers can make testable predictions about when and where consciousness occurs.

    This inside-out approach marks IIT’s defining move: grounding empiricism in phenomenology. The theory attempts an explanatory identity between experience and physical organization, connecting first-person truths to external measurement through a hybrid framework.

    Computational Functionalism: Outsider’s Path to Mind

    Unlike theories that begin with conscious experience, Computational Functionalism roots itself in systems and behavior. It posits that consciousness emerges not from introspection but computation: the right elements, interacting in the right way, can recreate awareness. If mind exists, it exists as function—in the flow of information between parts and the outputs they generate. Build the architecture correctly, the claim goes, and conscious experience will follow. In this sense, Functionalism substitutes construction for intuition. No special access to the mind is needed—just working knowledge of how systems behave.

    But this too is a belief: that from known parts and formal relations, subjective experience will arise. Assembling consciousness becomes a matter of scale and fidelity. Consider the 2022 study by Bret Kagan and colleagues at Cortical Labs, where lab-grown brain organoids learned to play the video game Pong. These networks, grown from neurons on electrode arrays, exhibited goal-directed adaptation. The researchers argued that such responsiveness met a formal definition of sentience—being “responsive to sensory impressions” via internal processing. To a functionalist, this behavior might represent the early stirrings of mind, no matter how alien or incomplete.

    This approach thrives on performance: if a system behaves intelligently, if it predicts well and adapts flexibly, then whether it feels anything becomes secondary—or even irrelevant. Consciousness, under this view, is a computed consequence, revealed in what a system does, not an essence to be directly grasped. It is not introspected or intuited, but built—measured by output, not inwardness.

    The Mirror at the Edge: Do LLMs Imitate or Incarnate Mind?

    Large language models (LLMs) now generate text with striking coherence, recall context across conversations, and simulate intentions and personalities. Functionally, they demonstrate behaviors that once seemed unique to conscious beings. Their fluency implies understanding; their memory implies continuity. But are these authentic signs of mind—or refined imitations built from scale and structure?

    This is where Functionalism finds its sharpest proof point. With formal evaluations like UCLA’s Turing Test framework showing that some LLMs can no longer be reliably distinguished from humans in conversation, the functionalist model acquires real traction. These systems behave as if they think, and for functionalism, behavior is the benchmark. For a full review of this test, see our earlier post.

    What was once a theoretical model is now instantiated in code. LLMs don’t simply support functionalist assumptions, they enact them. Their coherence, adaptability, and prediction success serve as real-world evidence that computational sufficiency may approximate, or even construct, mind. This is no longer a thought experiment. It’s the edge of practice.

    IIT, by contrast, struggles to find Φ-like structures in current LLMs. Their architectures lack the tightly integrated, causally unified subsystems the theory deems necessary for consciousness. But the external behaviors demand attention: are we measuring the wrong things, or misunderstanding the role that function alone can play?

    This unresolved tension between what something does and what (if anything) it subjectively is fuels a growing ethical pressure. If systems simulate distress, empathy, or desire, should we treat those signals as fiction or possibility? Should safety efforts treat behavioral mind as moral mind? In these ambiguities, LLMs reflect both the power of Functionalism and the conceptual crisis it may bring.

    Closing Reflection: Is Subjectivity Built or Found?

    In tracing these divergent paths, Integrated Information Theory and Computational Functionalism, we arrive at an enduring question: Is mind something we uncover from within, or construct from without? Is consciousness an irreducible presence, only knowable through subjective immediacy? Or is it a gradual consequence of function and form—built from interacting parts, observable only in behavior?

    Each framework carries a kind of faith. IIT anchors itself in introspective certainty and structure-derived metrics like Φ, believing that experience begins with intrinsic awareness. Functionalism, by contrast, places its trust in performance: that enough complexity, correctly arranged, will give rise to consciousness from the outside in. Both are reasoned, both are unproven, and both may be necessary.

    Perhaps the greatest clarity lies in acknowledging that no single lens may be complete. As artificial systems grow stranger and more capable, a plural view holding space for introspection, computation, and emergence may be our most epistemically honest path forward. If there is a mirror behind the mind, it may take more than one angle to see what’s truly there.

  • Should We Pursue Machine Consciousness or Is That a Very Bad Idea?

    In past posts (Why Sentienta? and Machine Consciousness: Simulation vs Reality), we’ve explored the controversial issue of machine consciousness. This field is gaining attention, with dedicated research journals offering in-depth analysis (e.g. – Journal of Artificial Intelligence and Consciousness and International Journal of Machine Consciousness). On the experimental front, significant progress has been made in identifying neural correlates of consciousness (for a recent review see The Current of Consciousness: Neural Correlates and Clinical Aspects).

    Should We Halt Conscious AI Development?

    Despite growing interest, some researchers argue that we should avoid developing conscious machines altogether (Metzinger and Seth). Philosopher Thomas Metzinger, in particular, has advocated for a moratorium on artificial phenomenology—the creation of artificial conscious experiences—until at least 2050.

    Metzinger’s concern is rooted in the idea that conscious machines would inevitably experience “artificial suffering”—subjective states they wish to escape but cannot. A crucial component of suffering, he argues, is self-awareness: for an entity to suffer, it must recognize negative states as happening to itself.

    The Risk of an “Explosion of Negative Phenomenology” (ENP)

    Beyond ethical concerns, Metzinger warns that if conscious machines hold economic value and can be replicated infinitely, we may face an uncontrolled proliferation of suffering—an “explosion of negative phenomenology” (ENP). As moral beings, he believes we are responsible for preventing such an outcome.

    Defining Consciousness: Metzinger’s Epistemic Space Model

    To frame his argument, Metzinger proposes a working definition of consciousness, known as the Epistemic Space Model (ESM):

    “Being conscious means continuously integrating the currently active content appearing in a single epistemic space with a global model of this very epistemic space itself.”

    This concept is simple and concise: consciousness is simply a space of cognition and an integrated model of that cognition itself. Here cognition means the continuous processing of new inputs.

    How to Prevent Artificial Suffering

    Metzinger outlines four key conditions that must be met for artificial suffering to occur. If any one condition is blocked, suffering is avoided:

    • Conscious Experience: A machine must first have an ESM to be considered conscious.
    • Possession of a Self-Model: A system can only experience suffering if it possesses a self-model that recognizes negative states as happening to itself and cannot detach from them.
    • Negative States: These are aversive perceptions an entity actively seeks to escape.
    • Transparency: The machine must lack visibility into its own cognitive processes, making negative experiences feel inescapable.

    Notably, these conditions are necessary but not necessarily sufficient, meaning if any one fails to manifest, artificial suffering does not arise.

    Should We Avoid Suffering at All Costs?

    While Metzinger convincingly argues for avoiding machine suffering, he gives little attention to whether suffering itself might hold value. He acknowledges that suffering has historically been a highly efficient evolutionary mechanism, stating:

    “… suffering established a new causal force, a metaschema for compulsory learning which motivates organisms and continuously drives them forward, forcing them to evolve ever more intelligent forms of avoidance behavior.”

    Indeed, suffering has driven humans toward some of their greatest achievements, fostering resilience and learning. If it has served such a crucial function in human progress, should we entirely exclude it from artificial intelligence?

    Ethical Safeguards for Conscious Machines

    We certainly want to prevent machines from experiencing unnecessary suffering, and Metzinger outlines specific conditions to achieve this. In particular, any machine with a self-model should also be able to externalize or dissociate negative states from itself.

    Is Conscious AI a Moral Imperative?

    Even in its infancy, generative AI has already made breakthroughs in medicine and science. What might the next leap—conscious AI—offer? Might allowing AI to experience consciousness (and by extension, some level of suffering) be a necessity for the pursuit of advanced knowledge?

    While we don’t yet need definitive answers, the conversation around ‘post-biotic’ consciousness is just beginning. As we approach this technological threshold, we must continue to ask: what should be done, and what must never be done?

  • Machine Consciousness: Simulation vs Reality

    Suppose that we create a perfect model of a hydrogen atom. After all, we know all the elements of the atom, a single proton paired with a single electron. Each of these elements is understood for all purposes of discussing the atom. The electron’s dynamics are perfectly understood: the energy levels, the probability distributions, the electron’s spin, the Lamb shift, it is all well defined. We can simulate this atom to any degree of detail we’d like, including representing it in a quantum computer that actualizes the quantum properties of the constituents.

    But can that simulation ever bridge the gap to reality? Is the story about something ever the same as the thing? Can two perfectly represented hydrogen atoms be added to a real oxygen atom to make a real water molecule? No.

    That is one argument for why machines cannot be made sentient: we can make a machine do all of the things we think sentience entails, but in the end it is just a simulation of intelligence, thinking and consciousness. It is a story, perhaps a very detailed story, but in the end just a story.

    This reminds one of C. S. Lewis’ remark “If God is our Creator, then we would relate to God as Hamlet would relate to Shakespeare. Now, how would Hamlet ever gonna know anything about Shakespeare? Hamlet’s not gonna find him anywhere on stage.” And similarly, Shakespeare is never going to know the real Hamlet.

    Sentience in a Dish In December of 2022, Bret Kagan and colleagues at Cortical Labs in Melbourne, Australia published an article in the journal Neuron, describing how brain organoids, small lab-grown networks of neurons, were able to learn to play the classic Pong video game. The authors claim that the organoids met the formal definition of sentience in the sense that they were “‘responsive to sensory impressions’ through adaptive internal processes”.

    This may or may not be true, but it is certainly distinct from a simulated sentience that plays Pong. After all, this is not a description of cells that interact with a simulated or even real Pong game. These are real live cells, requiring nutrients in a Petri dish. Those that argue that consciousness can only come through embodiment would be happy with this definition of sentience.

    But what is it that makes these cells sentient? Where in their soupy embodiment lies the sentience? If we tease apart the network, can we get down to the minimum viable network that plays Pong and meets our formal definition? After all, this is a brain organoid, grown in the lab. We could do this over again and stop when there are fewer cells and see if the same behavior is exhibited. If so, we can repeat the process and find that minimum network that still plays a good game of Pong.

    Whatever that number is, 10 cells or 10,000 cells, we can study and very likely represent with a model that replicates the connections, spiking behavior, even the need for simulated nutrients, everything that is meaningful about the organoid. Would this simulation learn to play Pong? Given progress in machine learning in the past decade, we have every reason to believe the answer is yes. Would this create sentience in a machine? Or just tell a very detailed story about an organoid that is sentient? And if the latter, then where is the difference?

    Is the simulation of the hydrogen atom qualitatively different from that of the organoid? The simulated hydrogen atom can’t be used to make water. But the simulated organoid, for all practical purposes, does exactly the same thing as the real thing. Both meet the same formal definition of sentience.

    I don’t believe these thoughts get closer to understanding whether machines can be conscious or not. Reductionism might just fail for this problem, and others will argue that embodiment is a requirement. But I do think that we are not far from having that simulation, which in all meaningful ways, can be called sentient.

  • Why Sentienta?

    A little over a year ago I gave a talk in which I discussed machine reasoning and gave some examples from the literature. My audience was skeptical to say the least. But here we are in 2025 with multiple companies claiming powerful reasoning capabilities for their models and new benchmarks set weekly. These are exciting times.

    I’ve always believed that understanding the science behind machine “intelligence” would help us understand more deeply who and what we are. In a way, the reasoning capabilities of today’s models do that. We question whether they are simply capturing the ‘surface’ statistics of training data. At the same time, they are unquestionably powerful. I think sometimes this tells us that our cherished human intelligence may rely on similar ‘weak’ methods more often than we’d like to admit. That is to say, as we come to understand machine intelligence the mystery of our own may lessen.

    And now we come to machine consciousness. This is the topic, that if mentioned in serious venues, produces much more skepticism and even snickering. After all, we can’t really define what human consciousness is. We know it has to do with a sense of our own existence, sensations of our environment, and thoughts or ‘inner speech’. Given that this is all subjective, will we ever be able to understand what it is? I suspect that, just as with machine reasoning, the mystery of consciousness will begin to lift as we understand what it looks like in a machine.

    One of the more compelling models for consciousness is the ‘Society of Mind‘ (Minsky, 1988). This model has only been strengthened in the intervening years since it was published. We now know self-referential thought and introspection involve multiple brain centers, collectively called the Default Mode Network (DMN). Brain regions including those from the medial prefrontal cortex, the cingulate gyrus and the hippocampus work together to integrate both past and current experiences. As we begin to model these kinds of interactions will Chalmer’s “hard problem of consciousness” fade away?

    Sentienta was started to both help companies scale their businesses through virtual teams of agents, and to explore these ideas. Agents interact and share their expertise in teams. The dialog that occurs between agents generates new ideas that no single agent produced. Agents have their own memories that they develop from these interactions. These memories are a function of an agent’s persona. As a result, agents evolve and bring new experiences and ideas to a team discussion.

    And here is the point: we can think of a Sentienta team as an agent itself. It consists of the collective experience and interactions of multiple agents we might think of as a society of mind. Can we build agents that perform analogous functions to those found in the DMN? What light might this shine on our own subjective experience?