This is part 6 of a 6 part series about the color spaces of the VFX pipeline. The first article, ‘Color Spaces of the VFX Pipeline’ traced an overview of the color path from the original scene in the real world through capture with a camera, storage space, workspace, to display space. Part 2 in the series, expanded on the subject of color in the real world. Part 3 expanded on the topic of cameras and how they perceive the real world and their effect on the images they capture for our visual effects. Part 4 expanded on the subject of how we store image data to avoid damaging those images and to future-proof feature films. Part 5 expanded on the subject of the impact of choosing a workspace and how it affects the quality of our images. Part 6, Display Space, will address the subject of display devices and their color spaces. Parts 1-5 and previously archived articles may be accessed by becoming an FX Ecademy member.
Introduction
Displays affect our work in visual effects in two separate ways. First is the workstation monitor that we all use to do our work – visual effects, CGI, motion graphics, digital paint, editing, and color correcting. Second and most importantly, our fine work ends up presented on some kind of display device so its characteristics and limitations affect the presentation of our work to the public. Since we work through and ultimately end up on some kind of a display, it would be good to have an understanding of them.
The current state of affairs is that for television work our workstation displays are reasonably close to replicating the HDTV color space, look and feel. This is not true for feature film work as our workstations fall far below the digital cinema projectors’ capabilities. But all of this is about to change. An awesome new color space is being deployed that not only approaches the limits of human vision, but that same color space will now be used for workstation monitors, broadcast television, and digital cinema. To understand the present and get ready for the future, read on.
A word about color science here. Color science is a formidable topic with complex components of optics, color theory, signal processing, and the wildly complex human visual system. So the color science in this article is, of necessity, over-simplified in order to tell a story that is actually useful to visual effects artists. Color scientists are advised to avert their eyes!
1: Display Devices Performance Standards
We will start by understanding the key performance standards that are used to measure the characteristics of a display device such as primary chromaticities and contrast ratios. We will then following three main types of displays involved in visual effects – workstation monitors, HDTV, and Digital Cinema – to see how their display characteristics stand up relative to each other. Then finally we cast our steely gaze into the future to see what is coming at us with the next generation of UHDTV and Digital Cinema 2.0.
⊕ Primary Chromaticity
The primary chromaticities of a display device are the technical specifications for the exact color of the three primaries – red, green, and blue – used to produce the color for the display. This was covered in Part 5 – WorkSpaces, but a quick recap is warranted here.
The CIE Chromaticity diagram maps all possible colors the standard human eye can perceive. The color of a primary is specified by its location in this diagram, and for sRGB and Rec. 709 displays (which share the same primary chromaticities), green, for example, is located at coordinates 0.30, 060. When all three primaries are marked out on the diagram they define a triangle of all the possible colors that can be generated. This is referred to as the gamut for that display. If you create a color outside of this gamut you are guilty of creating an “out of gamut” error and will be punished accordingly.
⊕ Transfer Function
The transfer function (also covered in Part 5) is the relationship between the incoming signal to the display device and its light output, and it controls how bright or dark the image will appear on the screen. A rough approximation would be the gamma, but the actual details are a bit messier. Again, the famous “quick recap” here.
We need a bit more than just “gamma” to describe the transfer function because gamma alone would entail the slope of the line going vertical as the signal approaches zero. Since the slope of the line represents the signal gain at that point, the vertical line represents a gain of infinity which is electronically awkward (i.e. impossible). The fix is a short straight line segment of a moderate slope near zero shown here as the red line in the closeup inset. When the straight line segment and the system response curve are blended together the result is very close to a gamma value, which is 2.2 in this example.
⊕ Brightness
The brightness of a display is, of course, how much light it can project towards viewers and is measured in units called “nits”, one of several metrics for measuring the brightness of a surface. If you hold a candle one meter from a square meter of a white surface you will get about one nit. While the measure of brightness is clear enough, its appearance to the eye can vary greatly depending on other factors such as the viewing surround and black levels.
⊕ Bit Depth
Some would call it color depth, but it means how many bits are allocated to each color channel. Today, 8 or 10 bits per pixel per color channel is the norm. The more bits per channel the finer the gradient steps from dark to light. A low bit count can introduce banding and in fact, if an 8-bit image is pushed a little too hard you can introduce banding in post.
Increased bit depth gets more important the higher the contrast ratio of the display simply because it “stretches” the bits over a larger brightness range so the steps between the bits can become visible. Note that a 10 bit (1024) display will have 4 times as many steps between black and white than 8 bits (256), and 12 bits (4096) will have 8 times as many steps.
⊕ Contrast Ratio
The contrast ratio is the ratio between the brightest white compared to the darkest black it can produce and is a key factor in having a bright and colorful display. The problem with the contrast ratio is that it is not well defined so monitor manufacturers devise clever ways to increase their scores. One such method is to report the dynamic contrast ratio. These can produce truly astonishing numbers of up to 1,000,000:1!
The trick to dynamic contrast ratio is to measure the whites with the display brightness turned all the way up, then measure the blacks with the brightness turned all the way down. You will never see these fantastic contrast ratios yourself unless you want to stand there with your finger on the brightness button while you watch a movie. The more meaningful contrast ratio is the native contrast ratio which means what is the brightest white and darkest black on the screen at the same time as with a large checkerboard pattern. So when you are being impressed with contrast ratios be sure to check which type you are dealing with.
Now here is the bizarre thing about contrast ratios – let’s say a mythical monitor’s black level measures 1 nit and the white level measures 1000 nits. Clearly, that’s a contrast ratio of 1000:1. If you want to double the contrast ratio to 2000:1 you need to raise the whites to 2000 nits, right? Doubling the brightness of a display device can be done, but will entail some engineering, more expensive components, and higher power consumption. However (here’s the bizarre part) you can also double the contrast ratio by lowering the blacks but just 1/2 a nit from 1 nit down to 0.5 nits. Suddenly we now have a 2000:1 contrast ratio simply by making the blacks blacker.
And this is the industry trend for improving the contrast ratio. Today’s monitor or HDTV does not have truly black blacks. Just watch some movie credits roll by and check out the black background. Not so black, is it? The reason is the LCD display is backlit and when the pixels are commanded to shut off completely to make black, some of the backlight still leaks through. However, as we shall see shortly, there is some exciting new tech coming that makes much better blacks.
⊕ Frame Rates
HDTV and digital cinema displays are spec’d to exact frame rates, which defines how many original images it can display in a second. I think of frame rates as coming in two flavors – reasonable frame rates like 24, 25, 30, and 60 fps (frames per second) and unreasonable frame rates like 23.976, 29.97, and 59.98. An explanation of the reasonable frame rates goes thusly:
- 24 fps – standard 35 mm film frame rate
- 25 fps – PAL (European) television frame rate
- 30 fps – a “rounded-up” video frame rate
- 60 fps – double the 30 fps standard
The unreasonable frame rates are a bit messier to describe. Starting with 29.97, it is the HDTV video frame rate standard inherited from color television that was developed in 1953. Before color, the first black and white TV standards were 30 fps clocked by 60 cycle household electricity, and Europe’s 50 cycle power produced PAL TV at 25 fps. But when color TV was invented the additional color signal was superimposed over the original black and white TV signal to maintain downward compatibility with the millions of black and white TV’s already out there. However, this additional high-frequency signal interfered with the audio signal so the most cost-effective fix was to slightly “de-tune” the video signal by one part in 1000 slower, from 30 fps to 29.97. It fixed the audio interference problem but the world of video has suffered ever since.
So that explains the tortured history of the 29.97 frame rate. The 59.98 frame rate is easy since it is exactly twice the 29.97 frame rate because in video there are actually 2 fields for every frame. The excuse for the 23.976 frame rate is that this is what you get if you map a 24 fps film to a 29.97 video signal which you must do to transfer a film to video.
One piece of advice when dealing with frame rates; don’t assume that when you see 30 fps on your camera or clip that it is actually 29.97 or vice versa. The 30 fps frame rate is to be taken literally and not confused with 29.97. Modern digital displays have no trouble coping with all of these different frame rates so they invariably support multiple frame rates. Which devices support which frame rates will be revealed shortly when we talk about each display device.
⊕ Refresh Rates
Workstation monitors are not spec’d to frame rates but rather refresh rates – what is the maximum times per second a new image can be displayed. The actual frame rate you get depends on the video card driving the monitor. The refresh rate is measured in Hz (Hertz) which is cycles per second, so a 60 Hz refresh rate means the monitor is capable of displaying up to 60 frames per second. Gaming monitors support higher speeds of up to 240 Hz, but you will pay extra for the speed.
⊕ Viewing Surround
A critical issue that affects the appearance of any display device is the surround, which is the lighting environment in which a given display device is supposed to be viewed. It is actually part of the specs for the display device and comes in three versions: a dark surround for movie theaters, a dim surround for watching HDTV in typical living room lighting, and a bright surround for workstation monitors in office lighting.
There are two reasons why the surround is so important. The first is that the eye adapts to the lighting in the room, and that ‘adapted eye’ profoundly changes the appearance of the display device. The second issue is that the room light also falls onto the display screen which fogs up the blacks. Raising the black level even slightly drops the contrast ratio. It has been said that if everyone in a theater audience wore a white shirt the resulting bounce light would considerably degrade the movie.
The images above illustrate how important the surround is. Starting on the left, viewing a theatrical image in its proper dark surround it appears bright and the colors vibrant. Raise the theater lighting to the dim surround specified for HDTV viewing and it looks dimmer and desaturated. Raise the theater lighting again to the bright surround intended for workstation monitors and you see how diminished the image would be.
2: Display Devices
Armed with the above dictionary of performance parameters, we can now visit each display device in turn and understand its performance characteristics. Once you understand the different color spaces of these display devices it becomes clear why an image must get a color space conversion to appear correctly on a different type of display.
⊕ Workstation Monitors
These are the display devices for our workstations for visual effects, digital paint, motion graphics, editing, and colorists. Editors and colorists will invariably have a second broadcast monitor so they can view their work in the intended display space of rec. 709 for television, but CGI and VFX artists typically have a regular sRGB workstation monitor to ply their trade. Fortunately, an sRGB monitor is not too different from a Rec. 709 HDTV. Powerful laptops have become an important addition to workstation towers and the good news is that their monitors work and look essentially the same as our desktop monitors.
The design for modern LED/LCD monitors has converged around a colored LCD (Liquid Crystal Display) display backlit by white LEDs. Older designs still use a fluorescent tube for backlighting. Each pixel has a cluster of 3 LCD cells to attenuate the backlight from dark to light, and each LCD cell has a colored filter to make red, green, or blue light. The back panel provides the light, the LCD cells control the brightness, and the colored filter gives it the color. Most use edge lighting with LED’s around the edge of the screen and light diffusers to spread it uniformly over the entire screen while a few use direct lighting with LEDs across the entire screen.
⊕ Monitor Specs
There is no single spec for monitor resolution, but they typically range from 1920 x 1080 to 2560 x 1440 or higher. They use the sRGB color primaries illustrated here which are shared with Rec. 709 for HDTV. The difference is that monitors have a lower transfer function (gamma) and less brightness than HDTV. The gamma is 2.2 and the brightness is generally around 250 nits but can go as high as 500. Contrast ratios for monitors are typically around 1000:1. Bit depth is normally 8 bits but 10-bit monitors are available. Refresh rates for workstation monitors range from 60 to 120 Hz but the actual frame rate depends on the video card driving it. The viewing environment is a bright surround and the viewing distance is 20 to 40 inches. Laptop displays typically share the same specs except for being a bit less bright and noticeably smaller.
⊕ HDTV
Today’s HDTV is where all of our episodic television and broadcast commercial visual effects are displayed. Their construction is the same as the monitors above with a backlight panel providing light, LCD cells providing brightness control, and the colored filters providing color. While HDTV shares the same color primaries as workstation monitors, what makes it different is that it is much brighter, has deeper blacks, a darker gamma, and therefore a greater contrast ratio.
HDTV also supports interlaced scanning where 60 fields per second are interlaced to generate 30 frames per second (more correctly 59.94 to 29.97 fps). Interlaced was originally the only scan mode available, so HDTV added a progressive scan mode to make a more sensible TV standard. Interlaced scanning allows fast action to appear smooth and avoids the stroboscopic effect where fast action looks ‘steppy’. One can still shoot and broadcast interlaced video to this day. Just hope they don’t give it to you to add effects!
HDTV can have special image processing on board to do things like upres low res signals, increase frame rates with motion compensation, deinterlacing, color space conversion, compression artifact mitigation, sharpening, and enhance color and contrast. In other words, HDTV, unlike workstation monitors, mess with your picture. Some models also include ambient light sensors to automatically adjust the brightness and contrast to the room lighting so the display’s appearance will be more consistent with different surround levels.
⊕ HDTV Specs
HDTV resolution is 1920 x 1080 (aspect ratio 1.78), has square pixels, and its color primaries are shared with sRGB as illustrated above. HDTV gamma is nominally 2.4 with a typical brightness of 500 nits but can be up to 1000 if you are willing to pay. The contrast ratio is around 5000:1 and bit depth is 8 or 10 bits. Frame rates are tightly spec’d out at 23.97, 24, 25, 29.97, 30, 40, 59.94, and 60 fps. The viewing environment is a dim surround, like the typical living room. The viewing distance should be about three times the height of the screen, i.e. 5 feet for a 40-inch screen or 7.5 feet for 60 inches.
⊕ Digital Cinema
The typical high-quality 3-chip digital cinema projector today uses a xenon lamp as a white light source, separates the white light into red, green, and blue with prisms, then sends each color to its own DLP (Digital Light Processing) chip with about a million micro-mirrors providing the per-pixel brightness modulation from dark to light. DLP chips are expensive so there are lower-cost designs that only require one chip but they are not as bright because they project the red, green, and blue colors sequentially rather than simultaneously. This can result in color strobing artifacts during fast action which are dubbed the “rainbow effect“.
The weak link in current digital cinema projectors is the xenon lamp. They are power hogs requiring up to 3000 watts, get dimmer with age, and are expensive to replace ($750+). The lifetime of a lamp is in the range of 500-1500 hours, but an insensitive theater owner might be tempted to leave the lamp in too long to avoid the replacement cost at the expense of a dimmer movie for the audience. Worse, some will even lower the voltage to the lamp to extend its life, which also results in dim projection. As an added excitement, xenon lamps are a high-pressure device at about 440 psi (almost 30 times atmospheric pressure!) so they represent an explosion risk, are packaged and shipped with an explosion shield, and are very scary to replace.
⊕ Digital Cinema Specs
Digital Cinema resolution is 2048 x 1080 (aspect ratio 1.89) and its color primaries are illustrated above as DCI-P3 which is noticeably larger than Rec. 709. The gamma is an honest 2.6 and the brightness is an anemic 48 nits – but in the dark surround of a movie theater, it looks great to the dark-adapted eye. It is a 12-bit system with a bracing contrast ratio of 2000:1 simply because its blacks are really black. Frame rates are 24, 25, 30, 48, 50, and 60 fps with none of the 29.97 foolishness of HDTV. The viewing environment is a dark surround and the viewing distance is based on what row you sit in.
We are warned not to watch HDTV in a dark room because of eye strain, but why is that when we routinely watch movies in the dark? The problem is that an HDTV screen is actually 10 times brighter than a movie screen in the same dark surround.
♦ If you would like to learn more about working with linear images check out my webinar Working in Linear!
3: Future Tech
Just when you thought you finally got your head around today’s display technology it is on the verge of another great leap forward. New technology and new specs are just around the corner to make our displays bigger, brighter, and more exciting than ever before.
Incredibly, the future UHDTV and future Digital Cinema share the same Rec. 2020 color gamut shown here. It comes with extreme new color primaries that push the boundaries of display devices to the very edge of human vision. In addition to wider color gamuts, UHDTV brightness and contrast ratios are also going way up so we have some new acronyms to learn.
WCG (Wide Color Gamut) refers to the Rec. 2020 gamut being wider than Rec. 709 or even DCI-P3, although some manufacturers will inappropriately refer to their P3 compliant monitors as WCG as well. True HDR (High Dynamic Range) is supposed to have a contrast ratio of at least 8,000:1. Most of its large contrast ratio numbers come from deeper blacks, but the brightness is also increased from the typical 500 nits of HDTV to much brighter 600 to 1000 for HDR.
⊕ UHDTV
Ultra High Definition TV (UHDTV) while often falsely touted as “4k” is an exact doubling of HDTV in both dimensions so it is not a true 4k. Early versions of UHDTV have been out for a couple of years using the current LED/LCD tech, new OLED, and micro-LED emissive screens that are just around the corner that emit light directly. OLED has incredible near-zero blacks raising their contrast ratios to nearly infinite with brightness levels of around 540 nits. The micro-LED screens are further off due to current cost constraints but they hold the promise of the brightest displays, purest color, and lowest power consumption.
Samsung’s QLED “quantum dot” technology is actually an upgrade of the current LED/LCD display designs. They replace the passive RGB filters used to generate color with active colored crystal semi-conductors that pass more light from the backlight giving them a wider color gamut and much greater brightness. QLED displays routinely produce 1000 nits of brightness and high-end models can even manage 4000 nits.
Although it is tricky to attempt to display a high-performance image on a low-performance monitor like most of us are using today, the illustration above attempts to convey the visual impact that HDR would bring to the display if you were to set the two monitors side-by-side. The HDTV on the left would look fine until you glanced to the right at the HDR image on UHDTV. Now you can never go back. Midtones like clothes and cars would look similar, but the blacks would be incredibly deep and the brights would really pop with amazing color purity. Keep in mind that these displays are designed for a dim surround so if the lighting is too bright the blacks will be fogged up and these incredible contrast ratios will drop.
⊕ UHDTV Specs
UHDTV resolution is 3840 x 2160, exactly double HDTV, and uses the Rec. 2020 color primaries mentioned above. The transfer function is actually the same as Rec. 709 so the effective gamma is also 2.4. As mentioned above, HDR contrast ratios are to be 8,000:1 or more, but that can be achieved either with a brightness of 1000 or more and normal black levels or a lower brightness with zero black levels. Bit depths will be 8, 10, and 12 bits but frame rates, viewing surround and viewing distances are the same as HDTV.
⊕ Digital Cinema 2.0
Digital cinema is constantly being driven by the relentless advances in home television. Today, the current Rec. 709 HDTV specs are noticeably less than the current digital cinema DCI-P3 specs. That is all about to change. As today’s TVs have brighter colors and higher contrast ratios than ever it is getting harder and harder for theaters to give you a better viewing experience than your home cinema. With UHDTV resolution and Rec. 2020 color primaries, theaters are going to find it tough to beat the home viewing experience so they have to up their game as well.
Exotic new display technologies are being developed that will dramatically increase screen brightness, color purity, and contrast ratios. Recalling that zero blacks are the key to high contrast ratios, cinema has the advantage that when they turn off the light the blacks are really black. To take advantage of that they are developing two projection and one emissive technology.
The three competing technologies for next-gen displays are – RGB lasers, laser phosphor, and LED walls. RGB lasers are the logical choice because the light can be made very bright with pure colors. The three lasers each go to a DLP chip like the current tech so they are essentially replacing the xenon lamp with lasers. Not only are lasers brighter but have lifetimes of 10,000 to 20,000 hours compared to a xenon lamp’s 500 – 1500 hours. The big hurdle, of course, is the price. There are already several laser digital cinema projectors on the market and it’s only a matter of time until they penetrate the installed base.
The laser phosphor system addresses the RGB laser cost problem by being less expensive at the expense of performance. Its brightness is only comparable to a lamp projector, but the big win is 30,000 hours service life and half the electric bill. It uses a single blue laser (big savings here!) to illuminate a phosphor surface which produces yellow light. Split the yellow light into its components of red and green then add some of the original blue laser and you have RGB.
LED screens made up of an array of LED panels are in development and a few have been installed in the field with stunning results. They are 10 times brighter than lamp projectors, have fabulous color purity, and have great uniformity across the face of the screen. Their problem is cost. One projector covers an entire screen, and for a bigger screen, you just purchase the next size up projector. However, with LED panels if you double the screen size you quadruple the cost.
High Frame Rate (HFR) movies are very feasible because of the speed and brightness of these projectors so if HFR catches on the new digital cinemas will be ready for them. Starting in 2012 Peter Jackson filmed The Hobbit: an Unexpected Journey at 48 fps. I screened it once at 48 fps and it was indeed stunning. James Cameron is a great fan of 3D features but the 3D effect breaks down during fast action at 24 fps so Cameron has been pushing for higher frame rates for years. He would like to see 48 and even 60 fps presentations. Modern digital cinema cameras and digital cinema projectors can handle these frame rates easily so look for more HFR projects in the future. However, a 100-minute movie will have 144,000 frames at 24 fps, but at 60 fps it will require 360,000 frames so there is a price to be paid.
⊕ Digital Cinema 2.0 Specs
Next-gen digital cinema resolution is an honest 4k at 4096 x 2160 (aspect ratio 1.89) and will use the Rec. 2020 color gamut discussed above. The effective gamma is 2.4 and the brightness will vary depends on the projector type and ranges from a low of 48 nits for lamp and laser phosphor projectors to the RGB laser projectors that can do 100 nits or more with contrast ratios of up to a million to one. It is a 12-bit system and supports frame rates of 24, 25, 30, 48, 50, and 60 fps. The viewing environment is the dark surround of course and the viewing distance depends on your seat.
♦ If you would like to learn more about ‘color’ then check out my webinar All About Color for Digital Artists!
Conclusion
Understanding the various display spaces and technologies is essential to understanding how our work will ultimately appear to the viewer. But the real question is how will this affect you? Life is a bit easier for digital artists working in television because all that is needed is a broadcast quality monitor to see their work exactly as it will appear in the field. With the current Rec. 709 specs even a plain sRGB monitor is a fairly close match. But when Rec. 2020 is deployed the old sRGB monitors will no longer cut it because the difference will simply be too great.
So television work will require Rec. 2020 monitors that support both HDR and WCG. But the big hit will be 8-bit production. We might get away with 8-bit visual effects for Rec. 709, but not for a 10 or 12-bit HDR display. These performance specs are more like feature film so maybe it’s time for you to think about working in linear floating point.
Feature film artists will also have to upgrade to HDR displays to have any hope of seeing things close to how they will appear in theater. One problem will be the surround as the work environment will be at least a dim surround while the theater has a dark surround that dramatically boosts contrast ratios and apparent brightness. We might compensate for working in the wrong surround by using ultra-bright displays to better approximate the appearance in the digital cinema.
The quality requirements for compositing visual effects shots will have to go up due to the new high contrast projectors with ultra-pure colors. When we composite a shot we work hard to make sure the layers visually match for contrast, hue, and saturation. Projecting the shot with high-performance projectors will tend to pull those layers apart visually because small differences between layers become exaggerated on screen. The bottom line here is that color control and color management will become more important than ever.
Be sure to sign up for our newsletter so that you don’t miss informative articles like this one!
Until next time, Comp On!
Steve
0 Comments