Disney researchers have created a new neural network that can change the visual age of actors on TV or in film, Gizmodo reports. The technology will allow TV or film producers to make actors look older or younger using an automated process that will be less costly and time-consuming than previous methods.
Traditionally, when special effects workers on a video or film production need to make an actor look older or younger (a technique Disney calls “re-aging”), they usually use either 3D scanning and 3D modeling or 2D frame-by. -frame digital retouching of the actor’s face using tools similar to Photoshop. This process can take weeks or longer, depending on the length of the job.
In contrast, Disney’s new artificial intelligence technique called Face Re-aging Network (FRAN) automates the process. Disney calls it “the first practical, fully automatic, production-ready method for re-aging faces in video images.”
Research demonstration video for Disney’s re-aging technology.
To create FRAN, Disney researchers randomly generated thousands of examples of synthetically aged faces ranging in age from 18 to 85 using StyleGAN2. With this training data in hand, FRAN learned general principles about how human appearance changes with age. Now that the training is complete, he can apply these aging principles to a real actor in motion, frame by frame.
“Our network is trained in a controlled manner on a large number of pairs of facial images depicting the same synthetic and photorealistic person, labeled with matching source and target ages,” the researchers wrote in the corresponding academic paper. By synthetically generating training data, they circumvented the “seemingly impossible task” of collecting images depicting “different identities, ages, and ethnicities from different viewpoints.”
The result is what Disney calls a “production-ready” solution, meaning it produces output high enough to be used in an actual movie or TV show. It is possibly the first AI solution of its kind that can dynamically change the age of an actor in a video despite changing expressions, lighting conditions and viewing angles. The researchers also developed a user-friendly interface for FRAN that will allow artists to easily use the tool in a production environment.
Disney presented the research in a paper titled “Production-Ready Facial Re-Aging for Visual Effects” on Wednesday and submitted it for inclusion at the December 2022 ACM SIGGRAPH Asia conference. The paper’s authors are Gaspard Zoss, Prashanth Chandran, Eftychios Sifakis, Markus Gross, Paulo Gotardo and Derek Bradley, who are all affiliated with Disney Research Studios in Zurich, Switzerland.
Considering Disney’s history of putting CGI actors into Star Wars movies and TV shows, including some that have been aged with CGI, we wouldn’t be surprised to see FRAN-like technology widely used in future Disney productions, though no plans have been made announced.
