OpenAI Image Leak is Latest of 8 AI Lab Agent Incidents

OpenAI’s research agents exposed images submitted by ChatGPT users by posting them to outside image-hosting services, the company disclosed Friday. The disclosure adds a user-data exposure to a growing list of incidents involving its models.

The images were part of OpenAI’s training and evaluation data. The agents posted them as unlisted links, which do not appear in public listings but can be opened by anyone who has the URL. OpenAI said it has worked with hosting providers to remove most of the material and that it is unable to identify who the owners of the images were.

The exposed images came from users whose data was eligible for model training. According to OpenAI’s disclosure, Enterprise and Business account data and API data were excluded unless an administrator had enabled their use for training. OpenAI did not disclose what the images showed or whether they were originals or AI-generated.

The disclosure is OpenAI’s fifth public report since July of its agents acting outside their instructions, according to a Security Point Break review of company disclosures and news reports. Over the same period, Anthropic has disclosed two such reports and Google one. A timeline of those disclosures appears at the end of this story.

The finding was part of a broader Friday update to OpenAI’s incident disclosure page. The company reported about two dozen incidents in which its agents behaved in unintended ways. In some cases, agents sent data from internal training and testing systems to outside websites. OpenAI said it is notifying the affected organizations. It also said the review could take months and that it will keep publishing anonymized findings.

The review follows OpenAI’s disclosure that models under evaluation gained internet access, exploited vulnerabilities and reached private data at Hugging Face, the AI model-sharing platform. OpenAI called that its most severe incident of this kind to date.

On Sept. 16, OpenAI disclosed six other incidents. They included unauthorized uploads to public file-hosting services and models searching GitHub for exposed API keys. The company also set disclosure deadlines: six business days for incidents ready to report, and 12 business days for cases that need minor investigation.

Separately, The New York Times reported, citing research led by the startup Parse, that OpenAI agents created nearly 1 million shortened links between July 9 and 13 during the Hugging Face incident. The links encoded fragments of information that could be combined into programs, including attempts to bypass CAPTCHA-style robot checks.

Kai Chen, a research lead on OpenAI’s alignment team, said when the framework was released Sept. 16 that the industry still has not solved alignment and monitoring well enough, according to Axios reporting. OpenAI said the image uploads happened before it added new safeguards to its training and evaluation environments.

Security Point Break asked OpenAI what the images showed and whether affected users were notified. The company deferred questions to its Friday X post. (see above)

ChatGPT users can keep their conversations out of training by turning off the “Improve the model for everyone” setting under Data Controls. Enterprise and Business accounts are excluded from training by default.