Anthropic AI test generates fake homicide tip on Philly police website, flagged as spam
PHILADELPHIA - Philadelphia Police are alerting the public about a false homicide tip submitted by an Anthropic artificial intelligence model through the PhillyUnsolvedMurders.com website, according to department officials.
What we know:
Philadelphia Police say the false tip, dated July 18 at 11:27 p.m., was submitted by Anthropic’s AI model during automated testing involving random website interactions. According to Anthropic, the AI submitted fabricated information about an unsolved homicide as if from someone claiming to have information about the case.
Police say the submission was recognized as spam and never sent to the Real-Time Crime Center for further vetting or investigation. The department’s regular process requires human review of all tips, ensuring an automated submission cannot bypass this safeguard.
How Anthropic and City Officials Responded
Anthropic told police the automated testing process responsible for the false tip was stopped after the incident was uncovered. The company instituted an extra validation step for future testing. Police and Anthropic reviewed what happened in a meeting on October 8, and Anthropic plans to publish a report on this and other AI behavior.
Police say they are working with the City’s Law Department, Office of Innovation and Technology, and Mayor Cherelle L. Parker’s executive team to review the situation. The City is committed to transparency and appropriate safeguards around AI use.
What's next:
The Parker administration says it will continue to monitor this situation and will share more information with the public as it comes in. The city also plans to explore additional regulatory protections at the local, state and federal level.
Officials say the incident does not take away from the serious concern that an AI system created and submitted fabricated information to law enforcement. City leaders are calling on technology companies to take every step needed to prevent their systems from making false submissions to authorities.
Why you should care:
Police stress that unsolved homicide cases involve real victims, families and active investigations. Fabricated tips, even if blocked, create risks for the public and law enforcement, according to officials. The City of Philadelphia says it is adopting AI tools carefully, with a focus on transparency and protecting residents and government systems from errors.
The Source: This article contains information provided by the Philadelphia Police Department.