In a bold stance against the proliferation of deepfake nude imagery online, a mother and daughter duo in the United States have stepped into the forefront of a battle against digital deception.
Their resolve stems from the distressing discovery that the daughter's fabricated images, along with those of other girls at Westfield High, were crafted through the use of artificial intelligence (AI), found The Guardian.
“I didn’t know how quickly it could happen, with just one image. That it can happen to anyone, by anyone, with the click of a button," Dorota, the victim's mother mentioned in an interview.
The shock prompted Francesca and her mother to take action against the perpetrators. Dissatisfied with their school board's response and the lack of existing laws to hold the alleged culprits criminally responsible, they embarked on a journey that led them to Washington.
Their advocacy, marked by appearances on cable news and involvement in the introduction of the Preventing Deepfakes of Intimate Images Act, aimed to legally hold creators of non-consensual, sexually explicit deep fakes accountable in the United States.
Yiota Souras, Chief Legal Counsel with the National Center for Missing and Exploited Children (NCMEC), emphasised the unique danger posed by apps that can create a large number of victims rapidly.
The NCMEC collaborated with the Mani family, working to determine if the images generated at Westfield High had further circulated online. While victims' parents received assurances that the deep fakes had been deleted, the exact number of affected students was not disclosed by the school.
The Westfield public school district initiated an investigation into the incident, providing counselling to affected students.
Dr. Raymond González, the superintendent, acknowledged the challenges posed by artificial intelligence and technology in schools and expressed efforts to educate students while establishing clear guidelines for responsible technology use.
“This emerging technology is becoming more and more accessible to individuals of all ages,” said Dr Michael Bregy, superintendent for the Beverly Hills Unified School District.
ClothOff, the app allegedly used in the New Jersey case, denied involvement and suggested it might have been a competitor app without providing evidence.
The Mani family's advocacy led to the introduction of the Preventing Deepfakes of Intimate Images Act in Congress. Dorota Mani attended a subcommittee hearing on addressing the harm caused by deep fakes, while another bill introduced by Congressman Tom Kean Jr sought to create labelling rules for AI content to enhance distinguishability.
Despite bipartisan support for these bills and their senate counterparts, the federal legislative process, focused on partisan issues like impeachment inquiries, is still in its early stages. Nonetheless, five U.S. states have enacted laws to curb the use of explicit deep fakes, with around 20 others introducing related legislation.
Francesca Mani, a Polish migrant who came to the U.S. for college, is a wealthy entrepreneur actively involved in advocating for legal measures against deep fakes. The mother-daughter duo's efforts garnered bipartisan admiration in Congress.
A spokesperson for Congressman Joe Morelle praised them for turning their trauma into advocacy, emphasizing the need for criminal and civil penalties to hold individuals accountable.
The situation in New Jersey is not isolated, as evidenced by the expulsion of students in a Beverly Hills middle school for creating AI-generated deep fake explicit images. Similar incidents occurred in Pinecrest Cove Academy and Issaquah, Washington, involving suspension and police investigation.
Francesca Mani acknowledged hearing from parents nationwide, with many reluctant to go public due to a perceived lack of action. Her direct message to those behind the app involved in her daughter's case was straightforward: "Shame on them. They're just making money."
However, she quickly shifted focus to the next phase of her campaign, targeting platforms like Apple, Google, and Amazon, along with financial institutions such as PayPal, Amex, and Visa, which she believes facilitate the prosperity of such technology.
In their pursuit of justice and awareness, the Mani family's advocacy underscores the complexities and challenges posed by emerging technologies and the critical role legislation plays in safeguarding individuals against malicious use. Their courage serves as a catalyst for broader conversations around the ethical, legal, and societal implications of AI-generated content, emphasizing the need for proactive measures to ensure a safer digital space.
(With inputs from Guardian)
Bob Blackman MP speaks during the event