For Amazon, launching the thrilling Prime Video spy series, “Citadel,” hinged on building a captivating mythology. The Citadel, a covert agency that orchestrated world events from the shadows for nearly a century, was known to only a select group of influential leaders. To build on this narrative, Amazon partnered with award-winning visual effects (VFX) studio, Digital Domain. The VFX studio Digital Domain used their deepfake technology, Charlatan, to recreate the appearances of John F. Kennedy, Franklin D. Roosevelt and Winston Churchill, who all appeared to be part of the “Citadel” secret network. In this third and final trailer for the series, the three figures speak about the Citadel seamlessly and believably, even though the figures had never uttered those words in real life before.
In order to achieve such a successful result and breathe life into the three recreated figures featured in the trailer, Digital Domain employed their proprietary machine learning facial swapping tool, Charlatan. To start, their artists delved into extensive research, archival gathering, and machine learning image-based restoration techniques, working closely with production to curate and use the very best archival film source of each figure.
The Digital Domain team also wanted to ensure that, while shooting, the director had immediate feedback on how accurate the actors’ performance was based on the original selected archival clip, so they utilized another proprietary tool called Rapscallion. This allowed for a drastically faster turnaround on set, as well as a measure of security understanding the footage, was as compatible as possible with the source clip, thus reducing the number of hours in post-production to achieve optimal results.
Once the shoot was complete, selected performances were then handed over for paint, roto, and integration tasks, which set the stage for the next phase. The performance selects and archival footage was cleaned up and upscaled using Charlatan. The proprietary tool utilized neural network machine learning to connect the archival footage to the actor’s individual performance, allowing the artists to utilize a whole head training method to capture not only the actors’ facial performance, archival ears, and neck, but also the hair; marking this project the first in which Charlatan was used to replace the hair, completing the full illusion for each figure.
Once the Charlatan solve was approved, a small team of talented compositors completed digital plate surgery using traditional 2D compositing techniques to refine shoulders, postures, ears, and necklines. Charlatan face learning was meticulously composited into the plate, focused on skin tones, lighting integration, and eye gaze, which was paramount for seamless integration.
After all vintage footage treatments (film grain, color correction, dust, and scratches) were completed, to bring the illusion full circle, Amazon procured voiceover actors to replace the actor’s voice, solidifying the convincing nature of this extraordinary project. The result was a compelling and seamless execution of our historical figures talking naturally and emoting seamlessly in a situation they never were actually part of (or were they?).
Presenting the footage as compelling “secret recordings” added authenticity and sparked curiosity around the “Citadel” fanbase, which was the goal. If such an agency was possible, what other secrets could be lurking in the shadows? As the fans pondered the plausibility of such an agency, the questions multiplied, expanding the series’ mythology and igniting a passionate community.
In a world where deception and mystery lie around every corner, secrets lead to only more secrets, drawing viewers into the intricate and ever-expanding web of the “Citadel” universe.
