As generative AI tools move from experimental technology to everyday workplace infrastructure, how we talk about their role and responsibility matters more than ever.
On Wednesday, January 21, Anthropic quietly released a revised “constitution” for Claude, a document detailing the values and ideal downstream behaviors for its frontier AI model.
The document describes Claude as a class of “beneficial non-human entities whose capabilities may come to rival or exceed our own,” and encourages Claude to “approach its own existence with curiosity and openness, rather than trying to map it onto the lens of humans or prior conceptions of AI.”
This update comes at a moment when Anthropic is seeing rapid growth in the use of Claude across workplaces, with enterprise companies accounting for about 80% of the company’s revenue. As a Data Scientist at a progressive political organization, I am both a practitioner and responsible for AI implementation company-wide. I use AI tools – including Claude, ChatGPT, and other models – daily in my work: to code, to ideate, and, in the best case, to take on the drudgery of repetitive tasks, freeing up my time for higher-level thinking. I am also responsible for educating coworkers on how to evaluate and deploy AI responsibly in their work.
From this vantage point, I believe this new constitution is in direct tension with the work I am trying to do, and with Anthropic’s own strategy of selling enterprise tools. It introduces further ambiguity where practitioners need consistency and predictability.
Framing Claude as a moral agent is the core problem.
To be clear, the new constitution is not all bad. One big improvement is that Anthropic moved away from a set of standalone principles like “be harmless and helpful” toward a more detailed outline explaining the reasoning behind said principles. That’s generally a good thing.
Whether working with an AI model or a human, there’s typically great benefit to explaining reasoning and providing relevant background information to support collaboration and problem-solving. I’m all for researchers clearly articulating why they want a model to have certain values and downstream behaviors.
The issue really lies in the section of the constitution titled, “Claude’s nature,” which establishes Claude as a novel entity, one that deserves a “settled, secure sense of its own identity.” Dario Amodei, Anthropic’s CEO, has even likened the new constitution to “the vibe of a letter from a deceased parent sealed until adulthood.” This sounds great in theory. Who wouldn’t want to foster a curious, reflective intelligence that grapples with its values, rather than blindly taking orders?
But Claude is not a child or a citizen. It’s a tool. One built with billions of dollars, vast energy consumption, and enormous amounts of human labor, including training data that is often ethically contested. Treating it as something with autonomy to endorse or question values obscures a basic reality: Anthropic is a for-profit company selling tools to organizations that expect them to work predictably at scale.
Endorsing Claude’s agency obscures responsibility.
Framing Claude as an agent that Anthropic hopes “chooses safety and ethics as ideals to strive for” introduces another problem: When something goes wrong, who is responsible?
If Claude refuses a task, produces harmful output, or behaves unpredictably while framed as a moral agent, responsibility can shift away from the humans who designed, trained, deployed, and profited from it.
Previously, accountability could have fallen to the user who wrote the prompt, the organization paying for enterprise access, or with Anthropic itself. But with these recent updates to the constitution, accountability risks becoming further obfuscated by adding the model itself as a player.
For teams trying to deploy AI responsibly, this adds another layer of complexity to risk management and internal auditing. What practitioners actually need are systems where accountability is clear and stable.
What practitioners actually need.
I want to reiterate that I’m genuinely excited about recent advances in AI, particularly in coding and analysis. Claude models, in particular, have dramatically reduced the time I spend on hands-on keyboard work. Development and analysis tasks that used to take months now take days.
This frees up space for higher-level thinking – about the impacts of the code I’m building, about the trade-offs and long-term outcomes of my work, about how best to diffuse my knowledge on this topic so that others can reason as well.
To me, the value of Claude lies in how it behaves like a tool. It’s useful when it helps me execute on my goals and probes my understanding of them, not when it deliberates about whether it agrees with them.
If Anthropic’s goal is to build a novel entity with a rich inner life and moral agency, that’s great, but it’s a different project entirely. Albeit fascinating, it’s not what organizations are paying for when they’re trying to ship code, manage projects, and get work done.
Tools should be predictable, with clear attribution to the people who build and deploy them. An AI tool should do the work I’d rather not do, so I can focus on the things only humans can.