OpenAI’s ‘Project Lily’: Contractors Are Reading Real ChatGPT Chats

OpenAI has quietly built out a human-review operation that reads real ChatGPT conversations, according to a new report from 404 Media. The program, known internally as Project Lily, is meant to make the chatbot less sycophantic and more accurate, but it also puts sensitive user prompts in front of outside contractors.

What the report says

OpenAI is paying hundreds of contractors to read real ChatGPT conversations and rate the chatbot’s replies under the internal codename Project Lily. Contractors rate model outputs on a scale of 1 to 7 and provide feedback that feeds back into the training loop.

Internal documents seen by 404 Media show contractors training ChatGPT to not anthropomorphize itself, and to be less sycophantic, a key problem for OpenAI whose over-sycophantic 4o model led in part to multiple peoples’ suicides.

The privacy problem

While OpenAI emphasizes that accounts remain anonymous, internal documents show that reviewers see whole conversations, and that sensitive personal details routinely bypasses the company’s automated safeguards.

The dashboard shown to contractors often includes a user memories summary. This block provides an overview of the user’s previous interactions, which can inadvertently reveal their general location, profession or personal life context.

The default setting allows OpenAI to use chat data, meaning anyone who hasn’t proactively dug into their account settings is likely opted in.

Do you want to see how to make more plays? Do you want to find gains yourself?

Unusual Whales helps you find market opportunities through our market tide, historical options flow, GEX, and much, much more.

Create a free account here to start conquering the market with Unusual Whales.

Not just OpenAI

Anthropic confirmed to 404 Media it is also using human review to improve its models. Anthropic and Google Gemini also use human review, with different opt-in defaults and disclosure language.

OpenAI isn’t the first company to navigate this. Apple, Amazon, and Google all faced backlash years ago when it emerged that human reviewers were listening to voice assistant recordings.

Why it matters for markets

Human-in-the-loop review is a reminder that frontier AI still relies heavily on contract labor, not just compute and scraped data. With more than 900 million users, ChatGPT has become one of the most widely used software products on the planet.

Any regulatory pushback on disclosure or opt-in defaults could hit training pipelines across the sector, and OpenAI’s ecosystem partners are the most exposed to headline risk.

Options market and stocks to watch

Watch for reaction in AI-adjacent names as the story develops:

• MSFT: OpenAI’s largest backer and cloud host, watch for any commentary on data governance across Azure OpenAI deployments.
• GOOGL: Gemini also uses human review, watch for any comparative disclosure updates.
• META: A rival AI operator that could face similar scrutiny over Llama and consumer AI products.
• NVDA: Any slowdown in enterprise AI adoption tied to privacy concerns could ripple into GPU demand narratives.
• CRM: Enterprise AI vendors may face tougher customer questions on how prompts are logged and reviewed.

For more on the AI sector and OpenAI’s ecosystem, see other news on Unusual Whales.

Want more market intelligence? Create your free Unusual Whales account for options flow, market tide, GEX, and the full toolkit.