Apple Retina Display

By now it seems that most people on the planet have heard of Apple’s latest iPhone, the iPhone 4 which was released today.  One of the many compelling features of the new phone is the Retina Display.  When Steve Jobs first invoked this term at the WWDC, my eyebrows were raised.  Being a retinal scientist, I was immediately skeptical of just what he meant by “retinal display”.  My mind immediately raced and I wondered if it might have been some of the interesting technology I got to see on my last visit to one of Apple’s technology development labs.  I will not say anything about that visit, but this Retina Display, a super high resolution display was new technology that I had not seen before.  Essentially it is an LED backlit LCD display with a *326* pixel per inch (960×640) display (John Gruber of Daring Fireball called this resolution display back in March) where each pixel measures a scant 78μm.    Though as you can see from these images of the displays I captured under a microscope, these pixels are not square.  Rather they are rectangular and while the short axis is 78μm, the long axis on the iPhone 4 pixel is somewhere in the neighborhood of 102μm. Update 07/23/10:  After discussion with some folks, including an LCD engineer, they have pointed out that pixels are measured from center to center rather than edge to edge, so I have changed the scale bars to reflect new measurements with a micrometer. Additionally, others have emailed me noting that if the black space surrounding the pixels is taken into account, the pixels are in fact, square.  So, the measurement of 78μm for the iPhone 4 is in fact 78μm from center to center of every pixel.  Also, Ron Uebershaer sent in screenshots I’ve included at the bottom of this post that he made in MATLAB which conceptually demonstrate that the pixels are in fact square.

I am including images below of the iPhone 1G, the iPhone 3G, the iPhone 4G and the iPad to show some perspective on pixel sizes.  The scale bar and my measurements are approximate as I was having a tough time in the lab tonight finding an appropriate calibration.  Nevertheless, this should serve as a useful metric for examining the relative pixel sizes and for making the point of whether Apple’s Retina Display is marketing speak and hyperbole or if in fact, Apple’s claims have merit.

 

As you can see from this image, the iPhone 1G pixels (each composed of a red, green and blue sub-pixel) measure approximately 150μm x 500μm.  Also note the blurryness of the image.  This was optimally focused, but the LCD panel itself is behind a non-bonded pane of glass with touch sensor on it leading to some image degradation.

 

As in the 1G iPhone, the iPhone 3G pixels are essentially the same size, though with a different contact location.  Again, these pixels measure approximately 150μm x 150μm and this LCD display has the same blurring issues that are present in the iPhone 1G.

 

This image of the iPhone 4G LCD is made at the same magnification as the 1G and 3G iPhones illustrating the substantially smaller pixel size in the iPhone 4G.  These pixels are remarkably small and if you look carefully, appear to be composites themselves where each sub-pixel is composed of its own sub-pixels.  I am not sure about this however and it may simply be an artifact of the construction.  Also note that there is very little distortion in the pixel images as the iPhone 4G has a bonded glass cover, eliminating the space in between the LCD panel and the touch sensitive glass surface.

iPhone1: ~150 x 150μm

iPhone 3G: ~150μm x 150μm

iPhone 4G: ~78μm x 78μm

So… the claim from Steve was that this display had pixels that matched the resolution display of the human retina.  Now, fan of Apple that I am, this struck me as perhaps a bit hyperbolic, so I figured I’d do some quick calculations to see where this claim fell.  Apparently I am not the first Ph.D. to wonder as another came out calling the bluff of Mr. Jobs.  Here is the deal though… While Dr. Soneira was partially correct with respect to the retina, Apple’s Retina Display adequately represents the resolution at which images fall upon our retina.

Essentially, this is a claim of visual acuity which is the ability of the visual system to resolve fine detail.  There are an awful lot of considerations to take into account when making such a claim such as contrast, distance, the resolution of the display and some metric of pixel size which gives you an estimate of visual resolution on the retina.  Claims of contrast ratios are notoriously flexible in a number of displays and will be influenced by a number of optical factors as well as the content being viewed and the black and color levels of the pixels as well as overall luminance.  Apple claims an 800:1 pixel ratio and I’ll take them at their word on that and focus on the claims of resolution here.

A “normal” human eye is considered to have standard visual acuity or 20/20 vision.  This means that a 20/20 eye can discriminate two lines or two pixels separated by 1 arcminute (1/60 degree).

The ability of an optical system to resolve fine detail requires minute spacing of optical detectors.  In the retina, there detectors are the photoreceptors.  Objects we look at at projected through the cornea and lens and imaged on the back of the eye on a plane that ideally lines up with the retinal photoreceptors.

Theoretically the limit of retinal resolution, say the ability to distinguish patterns of alternating black and white lines is approximately 120pixels/degree in an optimal, healthy eye with no optical abnormalities.  Again, this corresponds to one minute of arc or 0.000291 radians (π/(60*180)).  If one assumes that the nominal focal length of the eye is approximately 16mm, an optimal distance from the eye for viewing detail might be around 12 inches away from the eye which is reasonable to assume for someone viewing detail on their iPhone.

Dr. Soneira’s claims are based upon a retinal calculation of .5 arcminutes which to my reading of the literature is too low.  According to a relatively recent, but authoritative study of photoreceptor density in the human retina (Curcio, C.A., K.R. Sloan, R.E. Kalina and A.E. Hendrickson 1990 Human photoreceptor topography. J. Comp. Neurol. 292:497-523.), peak cone density in the human averages 199,000 cones/mm2 with a range of 100,000 to 324,000.  Dr. Curcio et. al. calculated 77 cycles/degree or .78 arcminutes/cycle of *retinal* resolution.  However, this does not take into account the optics of the system which degrade image quality somewhat giving a commonly accepted resolution of 1 arcminute/cycle.  So, if a normal human eye can discriminate two points separated by 1 arcminute/cycle at a distance of a foot, we should be able to discriminate two points 89 micrometers apart which would work out to about 287 pixels per inch.  Since the iPhone 4G display is comfortably higher than that measure at 326 pixels per inch, I’d find Apple’s claims stand up to what the human eye can perceive.

 

For reference, I am also including an image of the iPad LCD taken at the same magnification as the iPhone images above.  As you can see, the pixel size is actually much larger and herringbone shaped which is not uncommon in high quality desktop displays like say, the Apple Cinema Display line.

 

 

Update 03/02/11:  Carles Mitjá has an entry with a proceedings citation highlighting image quality expectancy here.  He has three beautiful images of a MacBook Pro 15″ display, an iPhone 4 display and a very interesting 24″ Apple Cinema Display.

Update 08/24/12:  Looks like this article has resulted in my being quoted in the NYTimes for an article on choosing computer displays.

Update 12/15/13: Linked from an NBC News article on 4k televisions, Enough pixels already! TVs, tablets, phones surpass limits of human vision, experts say.

 

164 Replies to “Apple Retina Display”

  1. Hmmm … research I’ve seen suggests that the cone density of the retina near the center is such that it can discern resolutions close to 30 line pairs / mm, if the image is projected directly onto it (bypassing the eye’s lens). Of course real world human vision never comes close to this, but there have been tests where subjects with good eyes, viewing high contrast backlit test targets, could discern up to 21 lp/mm. With ordinary photographic detail, people can discern up to 11 lp / mm, if the contrast is high and the lighting is bright. The new iPhone display is the equivalent of 6 lp / mm. This is very fine resolution, but it’s no surprise that people with decent eyes can see the pixels. And all of this suggests that Mr. Jobs’s claim is goofy.

    I can send you my sources if you’re at all curious.

  2. I can distinguish the difference between 300 and 600 DPI laser printing at about 8 inches… but i typically hold my iPhone at about 12 to 14 inches so i doubt i will be able to make out “jaggies” when i upgrade to V4.

  3. Doesn’t a “pixel” constitute a red, green and blue element? Wouldn’t this make the size over 3 times larger in that direction?

  4. > Hey Steve! My iPhone’s resolution isn’t really as dense as my retina can discern. -JD

    You’re holding it wrong. Just hold it 6 inches further away. Not a big deal. -Steve

  5. Outstanding research and reportage, very generously shared with us.

    Thank you, and thank you to all the astute commentators on this fascinating topic!

  6. This is a marvelous article, as it answers a lot of my questions about this “Retinal Display”. Thanks a lot. :D

  7. I thought “Retina Display” was a branding of the display technology in iPhone 4 and, as with most branding, is evocative of its crispness as opposed to technical accuracy of any sort.

  8. It should be trivial to test this.
    (1) Write a little program which displays 2 white pixels on a black background, in the middle of the iPhone4 screen, and lets you press one of two on-screen button according to what you see.
    (2) Make it randomly display either pixels above each other, like : , or beside each other, like .. , together with the buttons labelled “|” and “–” for you to choose, and keep a running count of whether your decision is right or wrong.
    (3) Run the program and see how often you’re right. A number significantly above 50% means you can distinguish the dots.

  9. Great article. I think there are two typos, which I’d normally ignore, but since they clouded my understanding, I’ll ask to have corrected.

    In the sentence
    “In the retina, there detectors are the photoreceptors.”
    I think “there” is incorrect (but maybe I’m not following the intended meaning)

    In the sentence
     “Objects we look at at projected through the cornea and lens and imaged on the back of the eye on a plane that ideally lines up with the retinal photoreceptors.”
    the “at at” I think should be “at are”.

  10. Thank you for explaining! I am, however, now blind while trying to read with the photos in the corner of my eye!

  11. There is no thing like iPhone 4G…its iPhone 4…just 4 not 4G.
    If your G stands for generation and not the model name, then iPhone 3G is actually 2nd Generation.

  12. Interesting analysis. It seems to more and more apparent that the Steve wasn’t exaggerating about the Retina display.

    It would be interesting to see how the iPhone 4 screen appears under the microscope with the various screen protection films that are available.

    I imagine all such films would cause some image degradation but showing how much and how the degradation varied among the different films would be revealing.

  13. It seems to me, the retina display and its sub pixel layout could be extended to a paralax barrier (auto stereoscopic) LCD – by applying a precisly aligned mask film with tiny stripes.
    Great news! So now we can watch 3-D movies on the iPad with openKMQ and may also watch them soon on the 4G without glasses ;-)

  14. Does a Nyquist sampling theorem apply to the retina? It seems like you’d have to be displaying at twice the resolution the retina can discern to avoid it.

    1. The cones in the human retina are backed on something resembling a hexagonal grid. In addition, they have a gaussian response curve with respect to light hitting it. Nyquist is hard to apply in this case without a lot of thought.

      1. Actually, cones are *not* organized in a hexagonal grid. Look into adaptive optics work in the living human retina and you are going to be in for a surprise.

  15. I-phone (1G-3G) LCD are Transflective displays. Pixels have a transmissive and a small reflective section (mirror like).

    To reveal the reflective section during photography of the pixel, ambient external to the back light must be illuminating the screen from the front side. Looks like in your pictures you are showing only the transmissive pixel section.

    I would like to see if the Ipad or new Iphone 4 also have this transflective section. Since AFFS technology has a variant called AFFS+ that has this reflective section.

  16. Leonardo – ar you sure? Every transflective display I’ve ever seen has had a single coloured pixel area and a backlight that happens to be reflective (or, alternatively, a partially-transmissive reflective backing with a light behind it). I can’t imagine a colour transflective screen that didn’t use as much pixel area as possible would be at all legible. Admittedly, the last such screen I used was a Sony-Ericsson P800, which is no longer exactly cutting-edge.

    I would expect the black pixel areas to be electronic connectivity and light blocks to stop bleeding between the sub-pixels (and spacer material). Pretty much every LCD with a conventional layout looks more or less like this.

    But I’m prepared to be wrong – after all, I doubted Apple would ever go to 960×640!

  17. Original Iphone LCD was Transflective, With a Large pixel area that is fully transmissive and a very small pixel section that is Reflective. The reflective area is expected to be seen when ambient light is entering the LCD . But on the article picture, the reflective section is not shown.

    Regards

  18. Leonardo – interesting. It looks as though there’s light reflecting off the inter-pixel wiring. My knowledge of LCD technology stops short of my being able to tell whether this reflection would be modulated by the liquid crystal (i.e. whether the area above the wiring has a controlled darkness); I’m used to S-IPS panels where I’m reasonably sure it’s not controlled (and I thought the iPhone 4 was S-IPS), but maybe AFFS+ is different; I’ve had trouble interpreting what little I’ve been able to find out through googling.

    I’m surprised that this area has a significant contribution to the final image, even in bright light (I’d expect the display to light up almost-black-on-black), but then I’ve always put my body between an LCD and the sun when I wanted to look at it.

    Historically, transflective LCDs have appeared grey under front-lighting, the whole pixel area has contributed, and you can read them with the backlight off under even fairly dim lighting. The trade-off is that they’re not as bright or vibrant as a purely back-lit display, or as reflective as a display that’s purely front-lit. If AFFS+ is a back-lit display that’s very slightly reflective, that’s interesting – nice to learn something new!

    I would, nonetheless, hesitate to describe any older generation Apple product as anything other than purely backlit – certainly not in the class of the other transflective devices I’ve seen. Then again, I don’t own an iPhone, so I’ve only the evidence of Google to suggest that Apple don’t expect you to use it with the backlight turned off, as is standard for what most people would call a transflective screen.

    Thanks for the information, though. (It doesn’t affect the pixel pitch in this article’s calculations, since Bryan should be considering the step between equivalent positions on adjacent pixels rather than the controlled subset of the pixel, but it’s interesting to see what techniques modern LCDs use.)

  19. This is a fascinating article Bryan. For me the biggest improvement with iPhone 4 is the screen. It is truly stunning… and after your detailed explanation I can see why. Thanks

Leave a Reply to William Cancel reply

Your email address will not be published. Required fields are marked *