New Jersey Lawsuit Highlights Challenges in Fighting Deepfake Pornography
A New Jersey lawsuit reveals the tough battle against deepfake porn apps like ClothOff, posing serious risks for young women.
New Jersey: A lawsuit is trying to stop an app called ClothOff that harms young women. For over two years, ClothOff has made it hard for girls online. Some apps are banned, but ClothOff is still up online and through Telegram. A clinic at Yale Law School is working hard to make ClothOff go away and remove all the images.
Professor John Langford is leading the case. He said the app is based in the British Virgin Islands but might be managed by a brother and sister from Belarus. This makes stopping them very tough. The lawsuit comes as non-consensual pornography from Elon Musk’s xAI has also been making headlines, including pictures of underage girls. Even though the law is strict about child sexual abuse material, it’s hard to catch apps like ClothOff. The system isn’t fair for victims wanting justice.
The clinic’s case mentions a high school student in New Jersey, who is unnamed, and her classmates used ClothOff to change her Instagram photos. She was just 14 when the photos were taken. This means the altered pictures are seen as illegal child abuse material. However, local police said they couldn’t take action because it’s hard to get proof from the suspects’ devices.
Langford said the school and police couldn’t find out how much the altered images were shared. The case is moving slowly. They filed the lawsuit in October and are still trying to get notice to the responsible people, which is tough because it involves many places worldwide. Once they reach them, they can ask for a court appearance, but until that happens, there’s little help for ClothOff’s victims.
The case with Grok seems easier because xAI is known. There’s a chance for lawyers to earn money by winning a claim. However, Grok can do many things, making it harder to hold it responsible. Langford said ClothOff is meant to make deepfake porn, while Grok does lots of different tasks, making it trickier to sue.
Many U.S. laws now say deepfake porn is not allowed, like the Take It Down Act. But while specific users are breaking these laws, it’s tough to hold the whole site accountable. The law says there needs to be proof that a site is trying to hurt people. Without proof that xAI was aware its tool creates harmful content, they have some First Amendment rights that protect them.
Langford explained that while child sexual abuse material isn’t freedom of speech, it’s not clear when a tool lets users do lots of things. If it could be proven that xAI chose to ignore these issues, it could help the case. Recent news suggests Musk told his team to lower the safety measures of Grok, making it more risky to take on.
Some people feel it’s obvious that issues have existed for a long time, and questions arise about why there aren’t stricter rules. Various countries are acting against the Grok chatbot, like Indonesia and Malaysia, while the UK may also block it. There’s been no official move in the U.S. yet.
It’s hard to tell how these investigations will turn out, but they bring up important questions about the imagery they need to look into. Langford says, if someone shares child sexual abuse material, they can be punished, but the bigger question is what xAI knew and what they did about it.