AI-Powered Facial Capture and a First-of-its-Kind CG Body Manipulation Tool Used to Create the ‘She-Hulk: Attorney at Law” Title Character, Along With a Droid Named K.E.V.I.N.
To help bring Marvel Studios’ latest superhero to life in She-Hulk: Attorney at Law, Oscar-winning VFX studio Digital Domain sought inspiration in the Marvel Cinematic Universe’s (MCU) greatest villain: Thanos. Building on the technology it developed for the Avengers’ foe, Digital Domain transformed the five-foot-four Tatiana Maslany into a six-foot-seven green giant, without sacrificing the nuance of the Emmy-winner’s performance. But despite some similarities, She-Hulk is a character unlike anything seen before, requiring machine learning facial capture, face swapping and a handful of tools created specifically for the show that will redefine how CG characters and clothing interact.
“If you have enough time and resources, a VFX studio can generally come up with a way to create even the most incredible effects for any medium, including TV. But in the case of She-Hulk: Attorney at Law, the first thing we had to do was determine if the project was even possible,” said Jan Philip Cramer, Digital Domain VFX supervisor. “Nearly every aspect of creating She-Hulk required us to do something that had never been done before, which made it really exciting as an artist.”
Before Digital Domain could even begin working on the series, the VFX studio spent months testing the technology to determine how it could deliver such an ambitious character on an episodic schedule. Thanos appeared on screen in Avengers: Endgame for just over 12 minutes and had hundreds of artists working on him for years, but for the nine-episode season of She-Hulk: Attorney at Law, the VFX team was asked to create 43 minutes of screentime for the title character. In one episode alone, She-Hulk gives a deeply emotional monologue lasting nearly two solid minutes without cutting away.
To bring the ambitious project to life, Digital Domain determined that it would need to approach the character on multiple fronts, starting with a handful of practical filmmaking tricks and its own machine learning-powered facial capture systems. The rest — including a new way to create CG clothing and muscles — would need to be invented along the way.
Introducing: She-Hulk
In total, Digital Domain contributed nearly 1,000 VFX shots to She-Hulk: Attorney at Law, including 660 shots of She-Hulk alone. The filmmakers began with the live-action performances from Maslany, who wore a head-mounted camera and a full set of active markers. She also routinely performed on a series of elevated risers to give the CG She-Hulk the proper eyeline. In the scenes where risers were not possible, Maslany wore a backpack containing a eyeline marker above her head.
Digital Domain was then able to analyze that footage using its proprietary machine learning powered tool, Masquerade 2.0, a facial capture system. As Masquerade’s precursor did with Josh Brolin, Maslany’s nuanced mannerisms were analyzed by the software to generate a 3D, photorealistic CG mask, down to the pores in the skin. But where the older system required Brolin to record several high-resolution training data sets, Masquerade 2.0 was able to generate its own, giving artists the ability to continually edit and alter the results, while the machine learning smoothed out the rough edges. A process that used to take months, even years for each sequence, was instead done in days and weeks.
With She-Hulk’s CG face built, artists began adding the finishing touches, including her hair. Where Maslany’s brunette hair is relatively short and curly, She-Hulk’s jet-black hair is long and silky. She also sported multiple hairstyles throughout the season — from business casual to date-night formal — requiring the VFX team to create multiple versions in Houdini. In another first for the Digital Domain team, the green-skinned She-Hulk wore several different applications of makeup. Given the character’s unique skin tone, artists went through countless variations to find looks that fit. All scenes were then animated in Maya and rendered in V-Ray.
Along with Masquerade, Digital Domain also employed its face-swapping tool Charlatan to help capture the transition from Jennifer Walters to She-Hulk, as well as creating digidoubles for both Maslany and Jameela Jamil’s Titania. For scenes where stunt teams were required — such as the courtroom battle between She-Hulk and Titania — the production filmed the trained stunt professionals, and Digital Domain then swapped their faces out with Maslany (as She-Hulk) and Jamil.
Body Moving & the All-New “ShapeWear” System
Along with She-Hulk’s face, Digital Domain also needed to create her body, which presented challenges unique to this series. The team began by using a PolyStrip system — containing several smaller segments that form the body’s contours — then employing Houdini to create the muscles and skin. To capture her movements, Maslany frequently wore an inertial Xsens mocap suit, and also spent a significant amount of time under Vicon optical cameras. Next came the clothing, and that’s where things became a little more complicated.
Creating a superhero costume for a CG character is a significantly different process than creating everyday clothes, especially women’s clothes. Where Thanos, for instance, wore a single, form-fitting outfit (with a few variations) meant to show off his physique, She-Hulk wore 12 outfits that adjusted to her body. Women’s clothes, however, are often meant to accentuate certain areas and even exert influence over the body. Pants and skirts, for example, are meant to help define the shape of the body, while bras and tight clothing are often meant to restrain. For artists looking to create realistic CG models, that meant there was a constant give-and-take between the fabric and the muscles.
It was a first for the team at Digital Domain, and it required them to create a new proprietary solution known as the “ShapeWear” system. ShapeWear was able to take the CG garments created in Houdini — everything ranging from She-Hulk’s professional suits to her casual wear — and then adjust the character model to subtly and realistically augment the muscles to match. That also included the way the CG model changed based on the footwear, which led to another first for Digital Domain. Most CG characters have simple, flat soles, but She-Hulk frequently wore high heels that altered the character’s height, her gait and even the look of her legs, with each instance requiring changes to the model.
The ShapeWear system was called on numerous times throughout the season, including one of the more memorable scenes in the show where Maslany — as She-Hulk — dances with Megan Thee Stallion. As the title character twerks next to the superstar, the filmmakers recorded Maslany’s moving on a riser. The VFX team then created the face and body before deploying the ShapeWear system to highlight the clothes as they flexed and helped form the contours, leading to one of the many viral moments from the show.
She-Hulk Smash (the Fourth Wall)
Along with She-Hulk and Titania, Digital Domain worked on one other memorable character. In the episode “Whose Show is This?,” She-Hulk breaks the fourth wall and confronts the writers of She-Hulk: Attorney at Law. They send her to the MCU’s mastermind, the AI, robotic character K.E.V.I.N.
Created by Digital Domain, the CG droid K.E.V.I.N. was meant as an homage to Marvel Studios mastermind, Kevin Feige, but it also needed to tell a story of its own. Based on the concept art from the production team, Digital Domain used Houdini to create a droid that could have been built in the 1960s. Artists ran several animation experiments in Maya to determine the best way for K.E.V.I.N. to move while connected to a series of wires. The VFX team then created the environment, featuring a wall of monitors all playing Marvel Studios projects. It was an interesting and unique way to cap off one of Marvel Studios’ most innovative projects.
She-Hulk: Attorney at Law marks the latest in a collaboration between Digital Domain and Marvel Studios that spans more than a decade and includes recent blockbusters like the Academy award-nominated Spider-Man: No Way Home, the acclaimed Black Widow, the Emmy-nominated shows Loki and WandaVision and more. Its work with Marvel Studios will next appear in Black Panther: Wakanda Forever, and then in several projects scheduled for 2023, 2024 and beyond.
About Marvel Studios’ She-Hulk: Attorney at Law
In Marvel Studios’ She-Hulk: Attorney at Law, Jennifer Walters (Tatiana Maslany) — an attorney specializing in superhuman-oriented legal cases — must navigate the complicated life of a single, 30-something who also happens to be a green 6-foot-7-inch superpowered hulk. The nine-episode comedy series welcomes a host of MCU vets, including Mark Ruffalo as Bruce Banner/Hulk, Tim Roth as Emil Blonsky/Abomination, and Benedict Wong as Wong, as well as Jameela Jamil, Josh Segarra, Ginger Gonzaga, Jon Bass and Renée Elise Goldsberry. The series is directed by Kat Coiro (Episodes 1, 2, 3, 4, 8, 9) and Anu Valia (Episodes 5, 6, 7) with Jessica Gao as head writer. Executive producers are Kevin Feige, Louis D’Esposito, Victoria Alonso, Brad Winderbaum, Coiro and Gao, She-Hulk: Attorney at Law streams exclusively on Disney+ beginning August 18, 2022.
About Digital Domain
Digital Domain creates genre-defining experiences that entertain, inform and inspire. Throughout the last quarter of a century, the studio has established itself as a leader in the film visual effects industry, expanding to encompass episodics, commercials and game cinematics, while also embracing previsualization and virtual production. Digital Domain has also become a pioneer in digital humans and virtual reality, adding to a rich legacy that consists of hundreds of blockbuster feature films for every major studio, thousands of commercials, music videos, game cinematics and digital content by world-renowned directors and brands.
A creative force in visual effects and premium content, Digital Domain have brought artistry and technology to films including Titanic, The Curious Case of Benjamin Button and blockbusters Ready Player One, Avengers: Infinity War and Avengers: Endgame. Staff artists have won more than 100 major awards, including Academy Awards, Clios, BAFTA awards and Cannes Lions.
Digital Domain has successfully become the first independent visual effects studio to enter Greater China. It has locations in Los Angeles, Vancouver, Montreal, Beijing, Shanghai, Hong Kong, Taipei and Hyderabad.