BLG Alert: Federal Judge Considers OpenAI’s Ability to Honor Privacy Commitments

Overview

Last month, a magistrate judge in the Southern District of New York ordered OpenAI to preserve all ChatGPT output logs for certain ChatGPT and Open AI API users—including logs users have “deleted.” (Order.) The order requires OpenAI to “preserve and segregate all output log data that would otherwise be deleted on a going-forward basis until further order of the Court.”

OpenAI has asked the District Court Judge to overturn this decision. OpenAI’s objection is fully briefed as of today (objection and opposition) and a ruling is expected soon.

The present order overrides ChatGPT users’ ability to delete past conversations, even users who bought the right to maintain privacy over their chats and to clear past chats. The ruling thus conflicts with OpenAI’s stated privacy commitments and user agreements, placing both OpenAI and its customers at potential risk of noncompliance with privacy and data minimization laws such as the GDPR and the CCPA, respectively, Europe’s and California’s online privacy rules. (See Open AI Declaration ¶ 2)

Impact on OpenAI Customers

While some commentators have described the order as catastrophic for OpenAI and the AI industry, the situation remains fluid. Much hinges on how the district court rules.  Even an order leaving intact the magistrate judge’s specific holding could mandate preservation of the data only as an interim measure pending further proceedings into what, if any, data can and must be produced.

In the meantime, the affected OpenAI customers—including paying users with contractual confidentiality protections—should assume Open AI is currently retaining their recently “deleted” data, even if only provisionally. OpenAI has said that it is securely storing the data and will not produce it to the plaintiffs unless specifically ordered to do so, according to an official blog post. (Post.) This suggests that, at least for now, users’ ChatGPT data is relatively safe.

If the district judge affirms the order, the situation becomes more uncertain. OpenAI has said it will resist further disclosure, though it has not yet detailed how. The magistrate judge has alluded to anonymization protocols, but the ability to anonymize and the potential scope and effectiveness of any anonymization remain unclear.

At minimum, we expect OpenAI will consider notifying affected users and offering them an opportunity to object before any data is produced. While this may be of limited practical value for users who lack the resources or legal knowledge to file objections, it would at least provide a path for users to protect their rights.

Implications Beyond AI

While of great potential consequence for the AI sector, the case could have a broader impact. The core question of private data preservation and use in lawsuits pertains not just to artificial intelligence but more generally to the extent to which users can rely on privacy laws and contracts to protect their data.

The logic of the magistrate judge’s order could potentially extend to other cloud platforms—including those of Google, Microsoft, and Amazon—opening the door to discovery requests to prohibit them from deleting customer data in lawsuits that have nothing to do with those customers.

As a result, anyone who relies on cloud services has good reason to closely follow the court’s upcoming decision.