A Brief Look at the Technological Advancements in James Cameron's Avatar
It's about that time again where we get a much-needed update from James Cameron on his still 16-months-out Avatar. The last update arrived back in late May when Cameron gave a speech at the Microsoft Advance '08 conference and talked about the technological advances they've made with Avatar. This week it was the Hollywood Reporter that talked with him further about some of the most amazing digital advancements you'll hear about, including live rendering of CG environments while shooting, an editing system called Facial Performance Replacement, as well as a camera system called Simulcam. We're still very far out from even seeing Avatar, but it's already one of the most advanced films in production and the technology implemented within it is all a testament to the genius of James Cameron.
"You have to make a good film that would be a good film under any circumstances," Cameron explains. And although we already know this, he emphasizes that "it is the most challenging film I've ever made." Cameron expects that by the time it's finished, Avatar will contain 60% CG animation featuring characters created using performance capture system, and 40% live action with visual effects imagery filling in the gaps. Although this may end up being the most amazing film we'll see visually, Cameron hasn't forgotten how to direct. "You have to put the narrative first. The reality is no matter how many (3-D) screens we get, you are still going to have a large number of people -- possibly the majority of people -- who see the film in a 2-D environment." They've already finished all of the performance capture with actors and are currently in the post-production process, which brings us to the technology they've invented.
Workflow: "The way we developed the performance capture workflow on Avatar, is we have our virtual camera, which allows me to, in real time, hold a camera -- it's really a monitor -- in my hands and point it at the actors and see them as their CG chartacters." This technology is something I've also seen at both DreamWorks Animation and LucasArts / ILM. Those two production houses also have a camera system that puts the operator in the virtual environment as if it is there; seeing this in person, it's obvious that it is where the future of filmmaking is headed. During the shoot, actors wear motion capture suits and a rig that takes an image of their face. "That is going though facial algorithms and going back into the camera as a real-time CG face of the character. You see it talk; you see the eyes move. It is pretty phenomenal."
Asset Management: "Once we've laid down a take, the take exists in the digital asset management system. It an be accessed at any time. Long after the actors have gone home, I'm still out there with the virtual camera, shooting coverage on the scene. I just have to play the take back. I can do the close up, the wide shot… I can even move them around on a limited basis. We relight it. We do all kinds of things." Now we might finally understand what Cameron is doing in the remaining 16 months, because as he works on the CGI and technology, he can at any moment go back and change the shot or the camera angle or any element of a scene. In essence, he'll be editing the film to perfection the entire time. "When you are doing performance capture, creatively it's very daunting. It's very hard to imagine what it will look like. But if you can see it, if you can have a virtual image of what is it going to be like, then you are there… I'm still doing a lot of virtual camera work on the film … on stuff that was shot six months ago."
FPR: Some experienced cinephiles might be familiar with the term ADR, which stands for Automated Dialogue Replacement, which is the process of replacing audio in the editing process to fix scenes that have dialogue changes or poorly recorded audio from the shoot. On Avatar, the system Cameron has developed is called Facial Performance Replacement and it's similar to ADR, but for CGI faces. "We changed the words and [actor Laz Alonzo] redid the dialogue. We didn't have to recapture (his body performance) and he didn't have to put the performance capture suit on again. We were just creating new words, and we were creating a new face." Again, referring to the idea that Cameron is simply editing the film for the remaining 16 months, this technology allows him to change dialogue or facial reactions without reshooting any of it.
Simulcam: As an extension of the workflow aspect above, Simulcam is the term they coined for the camera system that simulates the actual CG environment that they're shooting. "We're taking our virtual production toolset and superimposing it on physical production. We turned the set on the soundstage into a capture volume and turned the physical camera into a capture virtual camera, so we were able to integrate CG characters and environments into our live action." Again, this is a concept that's somewhat hard to understand without actually seeing it in person, but it's as if the CG world they've created can be seen by the actors and the director while they're actually shooting it with motion capture suits on. For example, Cameron describes: "We have people in flying vehicles, and I can see what is outside the window, fed in, in real time." If only the Wachowski's could've used this on Speed Racer.
Cameron's goal with the visuals in Avatar has been to create "one movie where the aesthetics of physical production and the aesthetics of virtual production are, to the extent that we could do it, pretty much it identical." From the sounds of it and from the descriptions of the technology they've invented, this definitely sounds like what we'll see next December. If you're salivating for more, be sure to head over to the full Hollywood Reporter article. For now we'll continue to wait until our next update from Cameron and potentially the first teaser, which I hope arrives by year's end. I've said it before and I'll say it again - we are in for one hell of a cinematic experience next December and even 16 months out, it's apparent that Cameron is advancing the technology of cinema so much that we may truly see a real revolution with Avatar.
Reader Feedback - 15 Comments
Andrew on Aug 9, 2008
that facial thing got improved. in the incredible hulk, edward norton had lots of green dots on his face so he could get the hulks emotion right.
Darrin on Aug 9, 2008
They could use this technology to merge Titanic with The Abyss, and then lead it right into Aliens. By 2050, this movie will be a reality. I called it here first, on Firstshowing.net!
DCompose on Aug 9, 2008
did you guys hear that they're making a Care Bears movie! I wanna drop like ten hits and travel to Care Bear land for like two hours. (15 hours acid time)
the rainbow warrior on Aug 9, 2008
wow, 5-10 years later they won't even need cameraman on the set anymore, who knows, maybe they won't even need actors since they can just bloody CG it, don't get me wrong, i am a CG artist, but it feels like the film industry is going to be nasty, takes away a lot of job opportunities, but nonetheless we do always benefit as a member of the audience.
andrew on Aug 9, 2008
If you guys would like to see practically every article regarding Avatar dating back as far as 2006, then you can check out the fansite I contribute to, http://www.avatarmoviezone.com Hopefully it will further interest those who are looking forward to Avatar, and potentially build interest in those not quite up for it yet. But the CG character in these films are still performed by real people, so at most, we won't have Cameramen. But unless movies are ALL CGI, I think cameramen will be needed.
Chris Groves on Aug 9, 2008
Lots of bashing. This is the guy who gave you T2 Aliens and Titanic. During shooting those flicks, did you prefer them to use plastic puppets as they did in the 70 instead of using the latest tech ? Im sure whatever new stuff Cameron uses will fit great.
Shige on Aug 9, 2008
Cameron did Terminator 2. The ultimate of Science Fiction. And not to mention the only director to get a good performance out of ARRRRRhnold. So please mr. Cameron. Give us moar
Buttons on Aug 10, 2008
Great news from Cameron ... I think for those who maybe don't get the idea ...you may watch the 2 disc ( Special Features ) of the movie Beowulf (2007) ... and you will understand exactly how the CGI is growing ... Explanation : In the movie Beowulf they shoot the entire movie in a room that contains small directed lights ( directed to the set ) ... and these lights are giving the distance of the dots that is over the actors face and body ... and by this you will have exact information about every dot over the actor and that give a prospective look and position of him ( I use an actor as an example otherwise they even shoot a horse in that room too ) 🙂 ... In the same time there is a ready modeled characters that will replace the actor ... and in that time you can see directly from the camera or from a monitor how it will look the model ... with all the emotions and movements ... and you can rotate and move the camera as much as you want without loosing details ... In thetime of shooting the only thing that was not 100% ready was the background ... but even that can be managed if the director know exactly what landscape is in the background he want ...... or how the light will be in the room to give the exact light because we are speaking about shadow too in the same time or in which direction the snow will fall stuff like that .... Of course all these mean a good CGI work before even shooting ... from modeling the characters and the atmosphere that around the character too ... and from there the director can see all the stuff as he wish in real time 🙂 By the way they use almost the same techniques for The Polar Express (2004) too ... and for sure you may know that both movies were directed by Robert Zemeckis .. 😉
Shero on Aug 10, 2008
And some twit here couldn't differentiate between James Cameron and Micheal Bay. I think his name fits: FlavoroftheDay.
RandyG on Aug 10, 2008
Geez the way Cameron is going Actors will actually need to get a real job.
Johnw on Aug 11, 2008
im still waiting for aquamannnnnnnnn
jont on Aug 11, 2008
You can never replace actors.
Buttons on Aug 11, 2008
When is he going to do Piranha 3?
Ed Norton on Aug 16, 2008
Hey Indian M. Night Shyamalan !!! Dont F***k with Iron Jim. Mr Jems Cameron's AVATAR will make a new age in cinema. Indian M. Night Shyamalan!!! change your goddamn anemi's name. take care of your silly actions Indian f**k!!!!
David on Aug 18, 2008
Sorry, no commenting is allowed at this time.