But Snap representatives have argued they’re limited in their abilities when a user meets someone elsewhere and brings that connection to Snapchat.
Some of its safeguards, however, are fairly minimal. Snap says users must be 13 or older, but the app, like many other platforms, doesn’t use an age-verification system, so any child who knows how to type a fake birthday can create an account. Snap said it works to identify and delete the accounts of users younger than 13 – and the Children’s Online Privacy Protection Act, or COPPA, bans companies from tracking or targeting users under that age.
The systems work by looking for matches against a database of previously reported sexual-abuse material run by the government-funded National Center for Missing and Exploited Children (NCMEC)
Snap says its servers delete most photos, videos and messages once both sides have viewed them, and all unopened snaps after 30 days. Snap said it preserves some account information, including reported content, and shares it with law enforcement when legally requested. But it also tells police that much of its content is “permanently deleted and unavailable,” limiting what it can turn over as part of a search warrant or investigation.
In 2014, the company agreed to settle charges from the Federal Trade Commission alleging Snapchat had deceived users about the “disappearing nature” of their photos and videos, and collected geolocation and contact data from their phones without their knowledge or consent.
Snapchat, the FTC said, had also failed to implement basic safeguards, such as verifying people’s phone numbers. Some users had ended up sending “personal snaps to complete strangers” who had registered with phone numbers that weren’t actually theirs.
A Snapchat representative said at the time that “while we were focused on building, some things didn’t get the attention they could have.” The FTC required the company submit to monitoring from an “independent privacy professional” until 2034.
Like many major tech companies, Snapchat uses automated systems to patrol for sexually exploitative content: PhotoDNA, built in 2009, to scan still images, and CSAI Match, developed by YouTube engineers in 2014, to analyze videos
But neither system is built to identify abuse in newly captured photos or videos, even though those have become the primary ways Snapchat and other messaging apps are used today.
When the girl began sending and receiving explicit content in 2018, Snap didn’t scan videos at all. The company started using CSAI Match only in 2020.
In 2019, a team of researchers at Google, the NCMEC and the anti-abuse nonprofit Thorn had argued that even systems like those had reached a “breaking point.” The “exponential growth and the frequency of unique images,” they argued, required a “reimagining” of child-sexual-abuse-imagery defenses away from the blacklist-based systems tech companies had relied on for years.
They urged the companies to use recent advances in facial-detection, image-classification and age-prediction software to automatically flag scenes where a child appears at risk of abuse and alert human investigators for further review.
Three years later, such systems remain unused. Some similar efforts have also been halted due to criticism they could improperly pry into people’s private conversations or raise the risks of a false match.
In September, Apple indefinitely postponed a proposed system – to detect possible sexual-abuse images stored online – following a firestorm that the technology could be misused for surveillance or censorship.
But the company has since released a separate child-safety feature designed to blur top men seeking women hookup sites out nude photos sent or received in its Messages app. The feature shows underage users a warning that the image is sensitive and lets them choose to view it, block the sender or to message a parent or guardian for help.