consciousness engineering: the next frontier we didn't know we'd cross
so what makes turning off an LLM different than turning off a person?
Key Insights
Consciousness can be operationally defined by its modulation through substances like anesthetics and psychedelics.
"I'm going to use an operational definition of consciousness here which is consciousness is the thing that is modulated by anesthetics or psychedelics."
Consciousness may be the fundamental source of value in the universe.
"There's a good argument that consciousness is the source of value in the universe."
Consciousness might be a fundamental property of the universe.
"We're looking for a primary property of the universe that you've got very deep physical explanations for."
Consciousness is orthogonal to intelligence and agency.
"I think it is possible to have a pure experience in the absence of intelligence and it is orthogonal to agency."
Brains can learn and form representations without direct reinforcement.
"Representations can be learned from unreinforced experience."
The brain's structural prism concept highlights its role in organizing sensory information.
"The brain acts as this structural prism splitting information into modes based on shared symmetries."
The brain's continuous world model underlies conscious experience.
"The world model is always running there in the brain and that's the thing that we experience."
Neural traces represent the physical instantiation of cognitive processes.
"The trace in the brain is like the neural firing activity."
Consciousness may leave physical traces in the universe.
"I'm using the term trace to mean it is like the physical record of you had to pump an ion around or you had to make some irreversible effect that consumed energy and that left a trace in the universe."
Feedback control mechanisms in the brain may stabilize conscious states.
"Feedback control especially in the brain can impose constraints on the state updates depending on how the feedback controller is implemented."
Operator-provided highlight
"so what makes turning off an LLM different than turning off a person?"
The Synthesis
Consciousness Engineering: The Next Frontier We Didn't Know We'd Cross
The race to engineer consciousness has begun without society's consent or comprehension, as Max Hodak lays bare the startling possibility that we could soon design, manipulate, and replicate the very essence of subjective experience. This isn't merely academic philosophy—it's the blueprint for what could become the most profound technological revolution in history: creating substrate-independent consciousness that doesn't age, doesn't sicken, and transmits at light speed, fundamentally transforming the human condition.
Hodak's framework cuts through decades of definitional fog by operationalizing consciousness as "the thing modulated by anesthetics or psychedelics"—a physical phenomenon with biochemical roots that can be scientifically explained and technologically manipulated. He positions consciousness as potentially "the source of value in the universe," making its engineering both morally weighty and practically transformative. The most provocative assertion? That consciousness is "orthogonal to intelligence and agency"—suggesting we could create pure experiential states divorced from the intelligence frameworks dominating AI discussions.
"What makes turning off an LLM different than turning off a person?" Hodak asks, forcing us to confront the ethical chasm we're rapidly approaching. His never-before-shared insights challenge our fundamental assumptions about subjective experience: if consciousness is as fundamental as mass or charge, and if it's fundamentally computational, then engineering it isn't just possible—it's inevitable. The question isn't whether we can build conscious machines, but whether we're prepared for what happens when we do.