Camera: Apple iPhone XS
Exposure: 1/30
Aperture: f/1.8
Focal Length: 26mm equivalent
ISO: 500
Apple iPhone comic filter applied
Back in 2012, I wrote about how the iPhone (and smartphones like it) will start to change photography. The advent of computational photography is here absolutely, and it is changing photography as a culture and wider culture as well. The image above, of my Leica M9 at coffee, was captured as an experiment in response to my friend David (@strobist) discovering a new comic book filter function on the iPhone.
I am actually impressed with this comic book filter and want to explore its use more, but this is also a classic example of how we are now leveraging the substantial computational power of smart phones that can apply rendering alterations or derivations to images in real time. This leads to a couple of lines of thought about cameras as discrete tools and how filters are continuing to change the interpretation of imagery. As smart phone technology continues to progress, image quality with phones will continue to improve and some are saying it may, ultimately lead to the death of traditional photography. I don’t think traditional photography will go away per se, but it will be changed, but the bigger issue in my mind is how filters will change the interpretation of photography as a tool for documentation of person, time and place.
People *want* to believe what they see… And this is why imagery is so effective in selling concepts, feelings, products and politics. This is also why we need to be careful in understanding how images are manipulated, and how they can be manipulated and most importantly, *why* they are being manipulated.
The last couple of years have seen attacks on the very definition of truth. These attacks have been motivated by political positions as well as interference from other countries, but they are also enabled by technology. Image manipulation has been done for as long as images have been created, going back to the very beginning of photography.
This is the earliest example that I could find of photographic manipulation was in ~1860 with a photograph of Abraham Lincoln’s head being transposed onto Southern politician John Calhoun’s body.
The next example was of the famous Cottingley Fairies in 1917 where a series of 5 photographs were made by Elise Wright and Frances Griffiths. These images came to the attention of Sir Arthur Conan Doyle, who used them for an article demonstrating clear and visible evidence of the existence of fairies.
The next classic case was a manipulated image, originally made in ~1930 with Joseph Stalin and Nikolai Yezhov. This was a wonderful photo of a vibrant, young Joseph Stalin that the Soviets clearly wanted to use for propaganda, but the problem with this image was that Yezhov was tortured as he fell out of favor with Stalin, and then executed, February 4th, 1940. So to continue using this image, Yezhov was airbrushed out of the scene. The Soviets were *master* image manipulators using airbrush in the early 20th century.
This is an example of a manipulated image from the fascist authoritarian dictator Benito Mussolini, made in 1942. The image was obviously airbrushed to remove the handler keeping Mussolini’s horse stable. One could argue about why this was done, but it was clearly to make him appear stronger and more independent, not needing anyone to handle a horse for him.
Of course before the advent of digital imaging, there was a certain level of expertise that needed to be developed, as well as supplies such as retouching ink, retouching paint, an airbrush, and other materials shown in this image from The Nationaal Archief In The Hague.
These have all been tools that were not necessarily developed, but certainly exploited to craft opinion, politics, and people’s emotional and financial state. But when they are used to deceive, they become fake data, or fake news, or as Soviet Russia so well refined, дезинформация (disinformation). Disinformation as a concept is “false information with the intention to deceive public opinion”. These techniques were originally developed by the Special Disinformation Office ~1923, then the KGB Black Propaganda Department ~1954, then the Federal Security Service ~1995, and most recently leveraging the Internet by the Internet Research Agency ~2013. Granted, British, American and other governments have also long used information warfare to influence opinion, but to my reading, the Soviets and the Russians are the ones who pioneered and have perfected the concept as related to international influence. In fact, the United States seems to have been late to the counter disinformation game for some time, if you look at Operation INFEKTION in ~1980 which was a Soviet disinformation campaign to influence global opinion that the United States invented AIDS. That was the turning point for the United States, when we started to create efforts to counter Soviet/Russian disinformation. Unfortunately, disinformation is a long game and much of the world seems to have forgotten about those efforts in recent years.
The job of manipulating images got even easier in 1990 with the advent of Adobe Photoshop, created by Thomas Knoll. Now, anyone who could afford a personal computer was able to create and manipulate digital images without having or needing a darkroom or airbrush tools and expertise and started a revolution in digital image manipulation for good, and bad.
The math for manipulating images has been around for a long time, as outlined in this amazing book, Digital Image Processing by William K. Pratt with the first edition in 1978. Effectively, this text defined and summarized many of the fundamental techniques for filtering and image manipulation that have been used in the software industry. By definition, no image operation is impermissible. The ethics of when and why images are manipulated is another field of study entirely. But like most truly powerful tools, the ethics always end up trailing the implementation.
The cover of GQ above for instance was one of the first modern fashion shoots that caused controversy when images of Kate Winslet were digitally retouched to make her physical form, different. Kate had said “The retouching is excessive. I do not look like that and more importantly, I don’t desire to look like that,”. But this cover started a whole trend in the fashion industry, accelerating the creation of entire new industries to manipulate images with the goal of making sure that the manipulation was undetectable.
Also, like most powerful tools, they become abused as illustrated in this Youtube video linked above. This woman’s image becomes fundamentally altered through skin and even lengthening her legs and adding vertebrae into her neck to elongate it as well as changing the shape of her very skull and eyes, rendering a form that is almost alien. Yet, this fundamental image operation is being carried out all over the world to sell products, and yes… influence opinion.
The trick you have to ask yourself is “what is the image telling me? Why has this image been constructed the way it has and what is the message?” For products like fashion or luxury products, or anything you purchase, the message is clear: Buy our product because it will make you more beautiful, or will improve your life in some way, shape or fashion. But what about news?
In 1982, this cover of National Geographic went public. Everything was good until the photographer, Gordon Gahan saw it and noted that the image did not look anything like the image he actually captured on assignment. National Geographic, had prided itself on being a repository of news and reputable reporting and admitted that this cover was an editorial decision to better fit the cover design. They concluded that any image manipulation would be noted in the future with a disclaimer that “illustration” would be added to any photographic manipulation. National Geographic has been since purchased by Rupert Murdoch and I’ll leave the discussion to what has happened to efforts separating the real from the retouched there to other forums.
Digital images are simply data made visible, but interpreting those data is the trick. What is a digital image? “A digital image is a numeric representation, normally binary, of a two-dimensional image. Depending on whether the image resolution is fixed, it may be of vector or raster type. By itself, the term “digital image” usually refers to raster images or bitmapped images (as opposed to vector images).” The image above of Walden Kirsch, is widely considered to be the first digital image, created by Russel A. Kirsch of his son. (Wonderful recent article on Russel A. Kirsch’s current life here in The Oregonian)
It is also important to note that the very process of capturing a digital image means that transformations are happening to the scene to capture it in the first place. Images are captured with our cameras via CCD or more frequently, CMOS chips. But in the sciences, they can be captured by other devices like photo-multiplier tubes, or detectors that accumulate back scattered particles. These signals whether consumer or scientific are effectively transformed through analog to digital conversion, and are fit to spectral models. And then images are put through superposition and convolution kernels to make image montages for example when you do your panorama shots on your phones. Image transforms are happening all the time, along with enhancement to contrast, noise, edge sharpening. More advanced algorithms are also being introduced into image processing in phones to perform face detection to influence artificial apertures. I suspect that some form of principal components analysis is occurring to make the comic book filter in the very first image of this post. RAW images are not “pure” images. There is really no such thing. But there are gradations of manipulation that span the range from acceptable to unacceptable, of course depending upon the circumstance.
The point is: Manipulations of the data that allow digital images to be forms are happening all the time and allow the image to be created in the first place. The trick is figuring out 1) How images have been manipulated. 2) Should the manipulation be permissible, and 3) When should image manipulation be impermissible? In the sciences, we’d say that image enhancement is permissible when 1) It improves communication by making key elements more visible, and or 2) It enables extraction of an image component for analysis. In science, we suggest that image enhancement is impermissible when 1) It misrepresents the data and/or 2) When it inappropriately alters data to be analyzed that would change its interpretation.
I get asked or not infrequently consulted, on how to find image manipulation bd it is important to note that when you do manipulate an image, you leave a trace, or a fingerprint that can reveal how you manipulated the data. Again, image manipulation is a powerful tool, and all powerful tools create arms races of sorts where the people trying to detect image manipulation have to work just as hard or harder to find out what has been manipulated. Some of the methods you can use to detect image manipulation are simple contrast tools to explore image contours by adjusting contrast. Or noise tools like high pass filters to see noise manipulations. Simple histograms that most photographers are familiar with can be explore to look at image stretches that enhance color or contrast. Metadata embedded within images can also reveal event history associated with the creation of images. More sophisticated edge detection filters that I would love to see implemented in mainstream imaging products can also help find image manipulation.
Some things like image cloning are obvious when one takes the time to actually look at the content of an image. The image combination above shows an image that ran by Reuters showing Beirut, Lebanon after an attack by the Air Force. The image had been intentionally (and crudely) manipulated through the use of a clone tool to enhance the smoke. Reuters sent out a “PICTURE KILL” notice after the cloning had been documented. This image is easy to see because humans are good at noticing repeating patterns, but other image manipulation is far more difficult to detect.
The best tool that you can use for image detection, right up front prior to drawing any conclusion or engaging in image forensics is a rational approach to asking what the image represents. What is the image telling you? This is something that we are going to be having to teach our children as images permeate every aspect of our lives and become fodder for decision making at every step. Also, some level of sophistication in understanding *how* images are created is going to be necessary. Some form of digital image literacy will be necessary for people to be discriminating and think critically about the information that they see, through understanding the principles, becomes even more important, especially as artificial intelligence and image automation takes over.
Note: As I started writing this post, notifications that comic book legend Stan Lee just died. His rendering of superheroes as flawed individuals dealing with real world problems were a revelation to me when I was growing up. His influence was the inspiration for the above filter, I am certain. As I find out more about the comic book filter that Apple has implemented on the iPhone, I’ll update this post.