Many of us are becoming more aware of our privacy online, and more concerned about it. How many of us have wondered whether it is possible to use the internet under complete anonymity, or how much of our personal information is already out there? How much is too much, and how vulnerable does sharing content online actually make us?
At snapWONDERS I have been leading the team analysing media content and metadata, with the focus on exposing privacy, copyright and tracking concerns. Metadata inside a photo can absolutely be used for tracking and forensic purposes. So imagine my surprise when I spotted something that looked highly suspicious — my early assumption was that Facebook was adding metadata that could potentially be used for tracking and other unconsented purposes. That would allow a method of tracking what happens to media shared outside Facebook, and inferring relationships from the way it is passed around.
The Facebook metadata
Under the APP13 metadata within the photo, carrying the signature “Photoshop 3.0”, I found tags added under the IPTC section called “Special Instructions”. The value started with FBMD, followed by at least 32 bytes of hexadecimal-encoded data.
The photo had originally been uploaded to Facebook, so my immediate thought was that FBMD could stand for Facebook Metadata. The content certainly did not look like anything you would legitimately encode as “special instructions”.
The concerns
Researching further, it turns out these concerns were raised as far back as 2015 by Stack Overflow user Patrick Peccatte, who asked about images uploaded to Facebook containing IPTC/IIM fields that were being added automatically during the upload process. Two tags were involved: “Special Instructions” and “Original Transmission Reference”.
This was elaborated further by Edin Jusupovic, a cybersecurity expert, in a Twitter post in July 2019, which stated that this enables Facebook to track photos outside of their platform with high precision, and to track who originally uploaded them.
The encodings are visible within the metadata, however Jusupovic warned that “if the technology is weaponised then Facebook could potentially track its users without zero proof”.
It leaves me wondering whether steganography will become more prevalent in media shared online. Steganography is the practice of concealing a file, message or image inside another file — in this case, inside the photos themselves.
The ability to attach a unique reference to a piece of content, and associate that with a person through an account or a device, provides a mechanism to learn patterns, infer relationships and perform behavioural analysis.
A worked example
Suppose I share an original photo online with my mother, and extra tracking metadata is added marking me as the source. She downloads it and emails it to my sister. My sister later shares that same photo from a second account on the same platform — one that neither my mother nor I know about.
Through that marker alone, the platform can infer a relationship between my sister, my mother and me, even though none of us are connected there in any visible way. That is enough to trigger further analysis of the relationships between everyone involved. The same approach could surface friends of friends, or indirect connections through clubs, organisations or points of common interest.
Used differently, the same marker could establish who originally uploaded a photo, which has genuine merit for settling ownership disputes. The difference matters: there is a real distinction between information used to protect ownership and information used for profiling. What makes it uncomfortable is when that intention is never conveyed to the people uploading the content.
Why this matters to me
Privacy, copyright and tracking in digital photos is the problem I work on. If you have found hidden metadata in photos worth talking about, or a phone or camera that behaves in a way that concerns you, I would genuinely like to hear about it.
In the meantime, you can run a photo through snapWONDERS to see the metadata, privacy and copyright concerns it carries for yourself.