BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

Rock Star Psychologist Steven Pinker Explains Why #TheDress Looked White, Not Blue

POST WRITTEN BY
Steven Pinker
This article is more than 9 years old.

Pinker is the author of ten books, including The Language InstinctHow the Mind WorksThe Blank Slate, The Better Angels of Our Nature, and most recently, The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century. It's no surprise, then, that he would have perhaps the clearest explanation of #thedress that took over the internet. Read it below.

Not since the days of Mitch Ryder and Monica Lewinsky has a blue dress aroused so much passion. A Tumblr user posted this photo and pleaded "guys please help me - is this dress white and gold, or blue and black? Me and my friends can't agree and we are freaking the f*ck out."

She was not the only one freaking out -- the puzzle has ricocheted around the internet and set off hundreds of comments and speculations, including judgments by a number of celebrities. Within minutes a dozen students in my introductory psychology course emailed me, asking for an explanation. I had to catch a plane, and at the airport bar overheard the barmaid and several patrons debating the dress. Here’s my best guess as to what’s going on.

The puzzle has nothing to do with what philosophers call the inverted-spectrum paradox (Is my red the same as your red?), which pertains to cases in which people agree—at least overtly—about the color they are seeing.

Nor does it have anything to do with rods and cones. The viewing conditions for the image are all well into the brightness range of the cones. The rods aren’t seeing the image at all.

And the two different percepts don’t seem to depend on the color settings of their monitor. According to the internet reports, two people can look at the same screen and still see the colors differently.

What it has to do with is lightness constancy and color constancy. The visual system is faced with the problem that any patch of light hitting the eye could be from a dark object illuminated by a bright light or from a light object illuminated by a dim light. That is, as far as the brain knows, a patch in the image at 100 brightness units could come from a black surface that reflects 10% of 1000 candles or a white surface that reflects back 90% of 111 candles. Nonetheless we generally see a snowball in the shade as white and a lump of coal in sunlight as black. This is the phenomenon of lightness constancy. On top of this challenge, a yellow-orange patch could come from a white cloth illuminated by a warm light (such as a tungsten or halogen light bulb) or an orange cloth illuminated by white light (such as the sun and sky on a sunny day). Nonetheless we usually see a white shirt under a floor lamp as white, not orange. This is the phenomenon of color constancy.

The brain achieves these constancies by using the overall range of brightnesses and colors in the image, together with its interpretation of the 3D shapes and arrangements of the surfaces, to “guess” the color, brightness, and direction of the illumination in each major region of the image. It then removes this illumination information from each patch, yielding a residual value for that patch which (when all goes well) tracks the inherent lightness and color of the actual surfaces in the world. And that’s the value that corresponds to our subjective experience of the surface’s color. As a result, the same patch in an image can look very different depending on the overall illumination, in particular, whether the patch is seen to fall in direct illumination or in a shadow. Edward Adelson’s checker-shadow illusion, in which we cannot see that the A and B squares are physically identical on the screen, shows lightness constancy in action: we perceive the squares with the lightnesses they would have in the world.

Likewise, the Rubik’s-cube illusion shows color constancy in action: the brown square on the top and the yellow square on the side are physically identical on the screen, but would have to come from brown and yellow tiles in the world, given that the latter is in shadow, and indeed we see them as brown and yellow. In sum: the brain compensates for shadows and other kinds of illumination, and forces us to see the lightness and color that a surface would have in the world, which are different from the physical values on the screen.

Digital cameras are faced with the same problems, which are called “Exposure” and “White Balance” rather than lightness constancy and color constancy. The algorithms in their chips are nowhere near as sophisticated as those in the brain (for example, they can’t take the 3D structure of the scene into account), but by using averages, contrasts, and extremes across the image, they are often pretty good at compensating for lighting, and they generally render whites as whites, yellows as yellows, and so on. But this works only as long as they are photographing a typical scene, one that’s mostly illuminated by a single light source, like the sun or overhead lights, and that has a typical assortment of light, dark, and colored surfaces in it.

But when the scene is atypical in certain ways, the camera can yield a picture that is over- or underexposed (failures of lightnesss constancy) or that has a color cast (a failure of color constancy). And that brings us to the dress. The dress scene is a nightmare for a digital camera. First, the lighting is radically mixed. On first look the dress appears to be in shadow, while the backdrop is illuminated by very bright and warm (yellow-orange) floodlights. Alternatively (it’s hard to tell) the dress may have been blasted by a brilliant lamp whose light was also picked up reflective surfaces in front of the woman.

Second, the lights in the backdrop are so much brighter than the shadows that they have maxed out the recording ability of the camera’s sensor, and many patches in the backdrop are “clipped” to pure blinding white, providing no information about the color of the illumination.

A third problem is that the dress appears to be made of a lustrous, shiny material, with glossy highlights that don’t behave like a typical surface. The exposure and color-balance algorithms in a camera generally “assume” that they are looking at better-behaved matte surfaces.

One way or another, the camera could not cope with this extreme scene. According to the most recent internet sources, It now appears that the dress really is blue and black:

This suggests that the camera “assumed” that the dress was in a deep blue shadow, rather than being bathed in a yellowish floodlight. It canceled out most of this blue, and compensated for the deep shade, and thereby washed out the dress to a near white. For the same reason, it rendered the black stripes, which in reality were just reflecting bright warm light, as a dark yellow. (Dark yellow with lustrous highlights is perceived as gold.)

Back to the human perceiver. Brains are so good at color constancy that they often succeed even when looking at a photograph of a scene rather than the scene itself. That is, the brain often compensates for color casts in a photo and allows us to see the colors of the objects in the photograph as they would be in the world. But only up to a point – sometimes the same atypical conditions that fooled the camera (such as mixed lighting and unusual materials) can fool the brain, too. The blue-dress photo is one of these marginal cases. Some viewers see the scene as the camera apparently did, interpret the dress as if it were bathed in a bluish light from the shade, and see its colors as white and gold. Others glean enough information from the image to correctly see the dress as illuminated by a brilliant warm light, and hence see the light-bluish portions as deep blue. By the same calculation, a dark yellow patch on the retina (if it came from a surface bathed in bright warm light) would have to be a dark neutral, to wit, black.

A bit of Photoshopping shows how this works. Photoshop allows the user to second-guess the camera and decide what’s really black and what’s really white. If I set a bright highlight on the dress to “white,” consistent with the camera’s guess that it is a white surface in deep shade, the photo is rendered the way the white-and-gold viewers see it:

But if I set it the deepest shadow to “black,” correctly nullifying the blazing yellow-orange of the actual lighting, the photo is rendered the way the blue-and-black perceivers see it:

There is still the mystery of why some people accept the camera’s assumption of shadow illumination and see the dress as white and gold, while others sense the floodlight illumination and see the dress as blue and black. It’s possible that the difference reflects different amounts of experience with harsh uneven lighting, with shiny dresses, or both. But I suspect the difference is arbitrary. When a person looks at a highly degraded or distorted image, their brains can suddenly lock on to one interpretation, whereupon they can no longer see the image in any other way. That’s what happens in the famous Jesus and Dalmatian images. Initially they look like a splatter of blobs, but once the Jesus or Dalmatian pops out, you can no longer un-see them and recover the first impression of a spatter.

It’s possible that when people first see the dress photo, with its unusual lighting and material, their brains struggle to interpret the color and intensity of the illumination, lock on to one of the two possibilities, see the material of the dress in the appropriate real-world color, and thereafter have trouble seeing it in any other way.

Steven Pinker is a Johnstone Family Professor in the Department of Psychology at Harvard University. He conducts research on language and cognition, writes for publications such as the New York Times, Time and The New Republic, and is the author of ten books, including The Language Instinct, How the Mind Works, The Blank Slate, The Stuff of Thought, The Better Angels of Our Nature, and most recently, The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century.