[Part 1/2] Using Distributed Learning for Deepfake Detection
Part 1 /2: State of the Art and Presentation of our Approach
Fabien Gelus
In this article, we present several facial manipulation techniques known as “deepfakes” and show why it’s important to improve the research on deepfake detection. We present the state of art of the deepfake detection datasets and algorithms, and introduce a secure, traceable and distributed machine learning approach for a deepfake detection benchmark using the Substra framework.
Photo by Rosie Kerr on Unsplash
Deepfake is a form of “synthetic media” in which a person in an existing image or video is replaced with someone else's likeness. Deepfakes leverage powerful techniques from machine learning (ML) to manipulate or generate visual and audio content with a high potential to deceive (def. Wikipedia). They can potentially become harmful when used for revenge porn, fake news (educational examples of Mark Zuckerberg, Barack Obama, Boris Johnson and Jeremy Corbyn deepfakes have been published in a prevention scope), hoaxes and identity fraud. Deepfake has even become a Github topic & a Reddit feed.
How are deepfakes generated?
The name “deepfake” is often used to design different facial manipulation techniques. The most common ones are “Identity Swap” and “Facial Reenactment” techniques. These terms were first introduced in the FaceForensics++ paper.
Originally, the term “deepfakes” was used as a pseudonym by a Reddit user who claims to have written the first Identity Swap algorithm using ML and inspired the open-source software deepfakes/faceswap (2017). Identity Swap is an image synthesis technique based on Neural Networks, used to “swap” the face in an existing video with someone else’s face.
A side-by-side comparison of videos. To the left, a scene from the 2013 motion picture Man of Steel. To the right, the same scene modified using deepfake technology. Source: Wikipedia
The basic steps for creating an Identity Swap are face detection, face learning and image reconstruction. To learn the faces, most of the algorithms use autoencoders: they are unsupervised learning models, which are used to learn compressed representation of data. In this process, images of both faces A and B are fed to neural networks called encoder and decoder to learn a compressed representation of the faces. The encoder is used to compress the faces in features (i.e. expression and orientation), shown in the diagram below as “Latent Faces”, and a decoder is learning for each latent face to reconstruct the original image. By swapping the decoders, the autoencoder can then generate a face for subject B with the same features of subject A
Illustration of an autoencoder used to create deepfakes. Source: Alan Zucconi
The use of neural networks is way more efficient in terms of quality and realness, compared to other more “graphic-based” techniques for Identity Swap such as MarekKowalski/FaceSwap, which consists of a simple face detector and image blending.
After Identity Swap, another major facial manipulation technique is Facial Reenactment. It is used to alter the expressions of a person by transferring the expressions of a source person to the target.
Source : BuzzFeed (Youtube video)
Different deep learning methods can be used to generate Facial Reenactment videos: Face2Face, NeuralTextures and LipSynch for instance.
To generate deepfakes, several tools are available online. We chose to only focus to open sources pieces of software as the code is auditable:
FaceSwap-GAN (2018) is one of the first Identity Swap tools to introduce Generative Adversarial Networks (GANs) in addition to the original auto-encoder architecture. The particularity of GANs is the discriminative network which evaluates the realness of the deepfakes generated and learns the model to generate more realistic deepfakes.
DeepFaceLab (2018) is an Identity Swap tool which offers several models of different size and depths that are tunable (e.g. HD and GAN) and multiple face extraction modes. It provides a CLI and a light GUI for learning preview and an all-in-one Google Colab notebook. It proves to have a better quality than the original deepfakes/faceswap repository (see the Paper for more details).
First Order Model (2019) can generate both Identity Swap and Facial Reenactment manipulations. It also provides a Google Colab notebook to quickly generate deepfakes without the need of a high-end computer (see the Paper for more details).
Examples of Identity Swap (top) and Facial Reenactment (bottom) videos generated with First Order Model
Other facial manipulations such as Attributes Manipulation ([1912.01865] StarGAN v2: Diverse Image Synthesis for Multiple Domains) or Entire Synthesized faces ([1710.10196] Progressive Growing of GANs for Improved Quality, Stability, and Variation), try not to “fake” someone else, but to change physical attributes (hair color, skin, etc.) or generate whole faces randomly.
Entire synthesized faces using GANs like StyleGan2 were recently used to create fake personalities to spy communities or post disinformation.
Why is the identification of deepfake essential?
We saw that with the development of deepfake generation open source tools and the free access to remote computing resources, anybody can create his own deepfake. Considering the ease of use of these tools, their possible malicious usages and the improvement of deepfakes’ quality over the years (cf. the first megapixel-resolution deepfake method by Disney), it’s important to focus the research on the deepfakes detection. With the reach and speed of information on social media, convincing deepfakes can quickly reach millions of people and have negative impacts on our society: Facebook, Microsoft and Google understood the need of a fast and automatic deepfake detector and have already mobilized resources in this domain.
How can we detect deepfakes?
To detect facial-manipulated videos, several datasets were made, gathering “real” and “fake” videos from different sources, with the objective of training ML models and evaluating detection algorithms.
Some datasets are collecting “real” videos from Youtube and then generate deepfakes with different methods: the datasets Celeb-DF (2020) and FaceForensics++ (2019) used deepfakes/faceswap with some optimizations in order to make higher quality deepfakes.
Some other datasets use hired actors to record their “real” videos dataset and then generate deepfakes with them: DF-TIMIT (2018) used faceswap-GAN, while Google’s DeepFakeDetection (DFD) (2019) and Facebook’s DFDC (2020) used closed source algorithms.
Other datasets such as Deepfake in the wild (2019) are completely made from deepfakes published on the Internet.
These datasets are often not directly made available to download. You need to contact the providers and accept some conditions (use it only for non-commercial research and educational purposes only, do not distribute, copy or host the dataset, do not incorporate it into any other dataset, etc.). This is one of the reasons why a distributed learning approach could simplify the process of accessing and gathering these datasets (you will find the details of our approach later on).
Each dataset makers tried to build new detection methods and compare them with the existing others on their own testset: this is how the idea of public benchmark platforms emerged:
The FaceForensics benchmark (2019) is giving a public non-labeled testset and let researchers submit their own predictions (only 1 submission allowed). The best algorithm has 91.7% accuracy. The implementation details of the top-ranked methods are not disclosed, except for LGSC-Lite (84.8% accuracy).
The Kaggle DFDC challenge (2020) has two leaderboards, one made from predictions on a public non-labeled testset and another made from a re-run of participants' code by the host on a privately-held test set. The top-performing model achieved 82.6% average precision on the public leaderboard and 65% on the private one. The common themes among the winners have been shared and the 1st winning model is available on GitHub.
At the same time, several papers and methods to detect deepfakes have been published and the literature seems to be flourishing:
Artifact-Based methods:
Methods using CNN:
CNN combined with RNN:
Auto-encoders
And even more.
We can also note that methods to prevent images from being used in deepfakes are emerging: [2003.01279] Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems.
However, all these detection models are trained on different datasets and use different test sets to measure their efficiency: there is a need for a real common benchmark of these methods.
Some open source tools have been released, with the aim of gathering datasets and models from different contributors (e.g. dessa-oss/DeepFake-Detection and zerofox-oss/deepstar), but didn’t gain popularity, certainly due to the lack of documentation and orchestration between dataset and model owners. This is why we thought about a new approach.
Why using a framework like Substra?
We propose to use a secure, traceable and distributed ML approach to train and test your detection algorithms, with the help of the open source framework Substra.
This approach offers several advantages for training and testing deepfake detection models:
We plan to gather a large diversity of deepfake datasets in order to be as representative as possible of deepfakes you can find “in the wild”. We can then provide the opportunity to train models directly on a Substra public instance “TestNet” (for educational purpose)
A public benchmark will be made in order to test the submitted models on a common secret and heterogenous testset, providing a real evaluation of their performance (giving certifications perspectives) and a non-biased leaderboard.
Thanks to the Substra framework, the datasets are not exposed to users and are only accessible by the submitted algorithms. In the context of a partnership between dataset holders, participants can create a Substra node to compute training or testing tasks locally (where the data is stored), and then choose to send back the resulting model or only the results of the test, according to the terms of use of their dataset.
For the datasets owners, this guarantees that:
their dataset can’t be used by malicious actors: the framework enables traceability of the ML tasks, so that we know exactly how and on which data the detection models were trained. It will also guarantee the reproducibility of the test score.
the testset used for a benchmark of the detection models is kept secret.
Conceptual diagram of the deepfake detector benchmark process using the Substra framework.
To illustrate our approach, we will detail a basic use case (related to the diagram above):
A user of the TestNet submits his Algorithm to the Substra public node. He wants to train his model (implemented in the Algorithm) and then test it on all the datasets available.
The local dataset is the dataset of the public node, made up of datasets that have been agreed to be hosted by Substra. It is split in 2 parts, a train set and a test set.
On the other hand, dataset owners are potential organizations who would agree to a partnership but would prefer their data to stay with them. They choose if their data can be used for training tasks and/or testing tasks. The datasets can be of different nature (e.g. different generation methods for fake videos) but would keep the same format (e.g. mp4 videos with a maximum size of 40 mo).
Here we represent a use case where a dataset 1 is only made available for training tasks and a dataset 2 is only available for testing tasks (the model is not sent back).
The model is trained on the local dataset (the train set part) and on the dataset 1. It is then sent back to the user, and submitted for the testing task. The trained model is tested on the local dataset (the test set part) and the dataset 2. The results are then aggregated and displayed on the public leaderboard.
Conclusion
We saw that deepfakes are becoming a real threat for our social networks and to the Internet in general, not only due to malicious usages like disinformation, identity theft, or online harassment, but also due to the ease of use of deepfake creation tools and the improvement of their quality over the years. We also saw that the actual state of deepfake detection datasets doesn’t permit a great representation of the deepfakes we could encounter on the web, and that there is a need of gathering disparate sources and creating a common benchmark for all of the detection methods.
That’s why we introduced a secure, traceable and distributed ML approach using the Substra framework, with the aim of creating a public testnet with multiple datasets and a common benchmark. We could then ally dataset owners and algorithms makers to improve the state of art in deepfake detection.
If you’re interested in the technical aspects of the framework and the Substra implementation of a deepfake detector,a second part of this article will be published soon. Subscribe to our newsletter to stay tuned!
Get in touch
Interested in getting hands on the deepfake detector?
Substra examples repository: https://github.com/SubstraFoundation/substra-examples
Need some help or additional information? Come chat on Slack!