What
Collective Constitutional AI is an experiment involving ~1,000 Americans to democratically draft a constitution for a language model. This constitution aimed to align the AI’s operations with publicly sourced normative principles, exploring how AI development can be influenced by democratic processes. The project’s outcome was a new AI system trained against this publicly sourced constitution.
Who
The project was a collaborative effort between Anthropic, a leading AI research organization, and the Collective Intelligence Project, which focuses on harnessing the wisdom of crowds for better decision-making.
Where & When
The project took place in the United States, with the public input process running in October 2023.
Why
By involving the public in drafting a constitution for an AI system, the project aimed to make the normative values guiding AI more transparent and democratically legitimized.
How
Key principles
- Democratic Engagement
- Transparency in AI Values
Functions
- Public Input Process: Using the Polis platform, an open-source tool for online deliberation, to collect and vote on normative principles from the public.
- Drafting a Public Constitution: Analyzing votes and comments to create a constitution reflecting a consensus among participants, then training a new AI model against this constitution.
- Model Evaluation: Comparing the performance of the model aligned with the public constitution against Anthropic’s in-house constitution to evaluate the impact of the public input.
- Outcomes: The performance of the models was similar across language and math understanding tasks, helpfulness and harmlessness, and political ideology reflection. However, the Public model exhibited less bias than the Standard model, especially in areas like Disability Status and Physical Appearance. This finding aligns with the public constitution’s greater emphasis on accessibility, suggesting that the public input process effectively influenced the model to reduce bias in these areas.
Business Models
Unknown, presumably funded by Anthropic.