Cinematic Lighting In Games
I recently finished playing Uncharted 4: A Thief's End. I've been a huge fan of the Uncharted franchise since playing the first Uncharted game on the Play Station 3. The story was engaging, the characters likable and believable. And yes, they even managed to make the game play fun as well. This last installment was a home run in almost every way. But what amazed me most (and had me constantly using Play Station's new screen grab function which is really cool) was how incredible the game looked, and how cinematic the entire experience was. Seriously, I found myself stopping every few minutes to take screen grabs of each new environment I found myself in. The detail, texturing, lighting and overall artistry in these environments would be pretty damn good for a feature film, never mind real time rendered frames in a game. The cinematic quality they achieved in this game was amazing. I think a major contributing factor to that, or at least what I noticed most (being a lighter), was how well the game was lit.
Feature film studios and game studios work in much the same way. Both have to create everything in the computer from scratch. Both require concept art, modeling, rigging, animation, lighting, etc. A lot of the process is very much the same. But there are also some huge differences as well. How are things different you ask? Well...
Games have 1/30th of a second to generate a single frame. A frame from an animated feature film will often take dozens of hours to render. And basically, for the purposes of what we're talking about here, that frame rate requirement is one of the biggest difference between games and features, and what drives many of the differences in how the final images are created.
A feature film will often custom stage, dress, animate and light each shot separately. The staging, set dressing and lighting will all often be tweaked on a per shot basis, and made to look as good as possible for that particular camera. Sometimes objects in a set will be cheated for a particular shot. Sometimes the lighting direction will be slightly changed from shot to shot to get the best possible image from that angle.
A game like Uncharted does not have the luxury of locked and/or predictable cameras allowing for cheats to be made on a per shot basis. In a game, there is no "shot". While a game like Uncharted will certainly be designed to direct and guide players in a certain direction, the camera essentially is the player, and can go anywhere the player can go, and look anywhere the player can look. It all has to look good all the time.
In both animated features and in games, the final images are created by a renderer. Some animation studios use their own proprietary renderer, some use a commercial renderer such as Arnold. Similarly in the game world, some gaming studios have their own proprietary game engine, while some might use a commercial game engine such as Unity or Unreal.
But, the render process is very different in games vs features. In animation, generally before being rendered, a shot is broken up into pieces. Characters may be rendered separately from backgrounds. Effects elements may also be broken out into separate layers and integrated back in later. Some studios will even render out different passes for each light. All this gets sent to a render farm consisting of hundreds, or even thousands of processors. After many dozens of hours, the final passes will be finished and then composited back together by compositors, where often times these layers will be further tweaked.
In a game engine, all this must happen in a fraction of a second. And there is no render farm with tons of super fast processors loaded with gigs of RAM costing many tens (or hundreds) of thousands of dollars. In the case of a console game, there is just the console. That $400 box sitting under your TV has to do it all. And only has 1/30th of a second. Usually lighting is one of those things that must be cheated and/or greatly compromised to ensure real time playback. A game simply can't compute all the complex calculations required for a realistic image and still play back in real time. The number of interactive lights is usually limited, and a lot of the lighting gets baked into the textures. But as processors increase in speed and power, those limitations are disappearing, allowing for more complex and richer lighting scenarios. When you look at the incredible latitude animation studios have with generating their imagery, vs the equally incredible limitations placed on games for creating theirs, perhaps you'll understand my awe at a game like Uncharted.
I've definitely been taking notice lately as game engines have really stormed onto the scene. The exponential growth of computing power is resulting in GPU's that can deliver better and better graphics at cheaper and cheaper costs. Some games, like Uncharted, have started to blur the lines between the gaming and cinematic experience. I have taken advantage of the free release of the Unreal game engine and have been playing around with it. While there are definitely some limitations, I'm pretty impressed with what is possible now. When you look at the images games are creating these days, the stories they tell, the amazing sound design and music scores (Uncharted also boasts an amazing score), they truly are beginning to rival the work being done by their feature film counterparts.
The other amazing thing about all this is how far they've come, and how fast they've gotten here. When comparing the visual look of the first Uncharted installment to this latest one, it is plainly clear what massive strides forward games have taken. And they continue to get better and better. The future of game engine as a story telling medium is very promising, and I think the possibilities it presents are exciting. Just take a look at this animated short rendered in real time with Unity. Soon, very soon, I believe it will be possible to create animated shorts, and even entire series or features using game engine renderers. But that is a blog entry for another time.
BELOW: Just take a look at the detail in the model for Nathan Drake. You'd almost think you were looking at a photograph. Pretty amazing stuff. For more about the tech and wizardry behind creating Nathan check out this tech demo video.
GALLERY: In game screen grabs I took while playing Uncharted 4