How Color From Nothing works

The second picture really was black and white. The color came from inside your eyes. It is called an afterimage.

Tired color cells

The back of your eye is lined with cone cells that sense color. Some respond most to red light, some to green and some to blue. When you stare at one color, the cones that sense it fire hard and get tired. For a little while they become less sensitive. Scientists call this adaptation.

Colors come in opposite pairs

Your brain does not read the cones one by one. It compares them in two opposite pairs: red against green, and blue against yellow. When one side of a pair is tired, the pair tips toward the other side.

Why you saw the real colors

The first picture had every color swapped for its opposite. The sky was orange, which works your red and green cones hard and your blue cones hardly at all. So the red and green cones got tired. When the gray picture appeared, they under-reported, the blue cones won, and the gray sky looked blue. The same happened everywhere at once. The gray picture gave the light and dark, and your tired eyes painted in the real colors.

It only works while your eyes stay still. The tired patch moves with your eyes, so one look away and the colors slide off the picture and fade.