Dad sends naked picture of his child via Gmail to a doctor & his Google account gets blocked for good.
Machine-driven scanning in Gmail flagged an account after sending nude pictures of a child to the doctor. Now the account is blocked forever.
The incident described above happened during the COVID pandemic with many doctors’ practices being closed. The New York Times first reported about it.
The father noticed abnormalities in the genital area of his child, he sent photos for remote diagnosis at the request of the pediatrician’s office, as reported by the “New York Times”. Since he sent these pictures via Gmail, Google had access to them and scanned them for potential criminal activity.
Google’s algorithms detected the nude images, which immediately locked the account. The case was then forwarded to the authorities who started an investigation, but soon closed it as the father had not committed any crime. The police had access to all the information Google had on him and decided it did not constitute child abuse or exploitation.
Regardless of this decision, the Google account of the father remained locked because of sharing these naked pictures.
Even when the father appealed his case to Google again, providing the police report that declared him innocent, Google refused. Then his account was being permanently deleted due to inactivity.
The Google account was gone for good.
What does this mean for every Gmail user?
TL;DR: Beware: If you send naked pictures via Gmail, your entire account can get blocked.
Google has an automated tool to detect abusive images of children: The AI scans every Gmail message for abusive material, such as pictures of nude children. The system is very broad so even if you share naked pictures of your children at the beach or in the bathtub with a family member, the system might flag your Google account for sharing child pornography.
But when the system gets it wrong, the consequences are serious.
Your account can get locked for good; meaning there will be no possibility to unlock the account again.
Given that your Google account, and in particular your email account is very valuable, this is a severe risk. After all, your email account is the gate to your online identity: You connect lots of services for password resetting purposes to your email account, such as Amazon, Twitter, PayPal and more.
That’s why it is crucial to choose a secure email provider, one that protects your data and your privacy. At Tutanota we work hard to make sure no one can spy on you by encrypting all your data by defaults. Check out the benefits you get with encrypted email.
How does Gmail detect naked pictures of children
Google runs an automated system to scan every message you send for potential child sexual abuse material (CSAM).
To date, this system is voluntary, but the EU wants to make client-side scanning for CSA material obligatory which would be a devastating intrusion into everybody’s privacy.
This scanning process enables Google to flag potential criminal activity on its platform. However, the system is rather broad so that any picture of a naked child can get flagged.
What happens if Google flags you as a criminal
When Google detects criminal activity in your account (such as sharing a picture of a naked child), it immediately locks your entire Google account. This includes everything in Google: Gmail, Google Drive, Google contacts, everything.
You will immediately lose access to all your data, your contacts, your pictures, and more - without an option of regaining access.
Google will then report your profile to the authorities and if presented with a search warrant hand out all data requested about you.
This can even happen to you if you are completely innocent, as the case reported by the New York Times demonstrates.
Will Google unblock you after being flagged?
No. While the authorities quickly came to the conclusion that the father was innocent, Google refused to unlock his account. It was gone for good.
While the father was able to clear up the misunderstanding with the police who stopped their investigation, Google did nothing.
Since the process took a lot of time, the disabled account of the father got deleted in the meantime because it had been inactive for too long. As a result, the father lost a lot of important data, including years of family pictures.
The main problem here is obvious: Due to the practice of AI-driven scanning for criminal behaviour which automatically flags and disables abusive accounts, the accounts of innocents get blocked. And there is no option for them to regain access to their Google accounts.
In the New York Times article law professor Ms Hessick speculates: “From Google’s perspective, it’s easier to just deny these people the use of their services. Otherwise, the company would have to resolve more difficult questions about “what’s appropriate behavior with kids and then what’s appropriate to photograph or not.”
Keep your data local & pay for services
Many Google users report how difficult - or impossible - it is to get a locked account unlocked again (1, 2, 3. Apparently, customer support is not easy to reach, especially since they can only inform and not provide active assistance anyway.
Even if the father was actually in the right in this case, his example shows how “easy” it is to lose ones access to Google. It is therefore advisable not to use Google as your only back-up or original storage.
Instead you should always keep a local back-up of valuable data. It is also recommendable to pay for online services as direct support is often limited to paying customers. If you lose your login credentials, fast access to support can be crucial.