Post FeedThe most evil feature ever conceived: the Exif Orientation Tag
There are some advances in technology that are actually steps backwards: features that look nice on paper, but always get in the way when implemented in reality. One of my pet peeves in this category is the Exif Orientation Tag, a little flag present in JPEG files generated by digital cameras that causes all kinds of havoc. It’s one of the places where the old proverb »the road to hell is paved with good intentions« holds true, because the idea behind this flag is a good one, whereas the flag itself is a product of pure evil. But let’s start the story at the beginning …
The world before the Orientation Tag
The first digital cameras simply captured images in their default (i.e. landscape) orientation, period. If you held the camera in portrait orientation, you got a JPEG file with an image that was rotated by 90 degrees. Most image viewers can rotate images temporarily for display, but if you wanted to permanently fix the image to appear in its correct orientation, you had to modify it manually.
When done the naive way (using standard image editing software), this means that the JPEG file is decompressed, rotated and compressed again, which is a lossy process. However, due to the way the compression works, it is possible to rotate JPEG images losslessly (at least if both width and height are divisible by 16 pixels; if not, a few pixels may need to be cropped off at the edges). Such algorithms are implemented in most image viewers, and they already were back in the day.
So, to conclude, portrait photos from early digital cameras required manual conversion to rotate them into upright position. Since tools for that were already well established, this was not a serious problem, only a minor nuisance. All could be well if the story ended here.
Orientation Sensors and the Exif Orientation Tag
Some camera manufacturers wanted to solve the problem with portrait photos and thus integrated orientation sensors into their products. If the sensor detects that the camera is held in a portrait orientation, the image would automatically be rotated so that this doesn’t have to be done manually later on.
However, there was a small technical problem: The image signal processing chips (ISPs) in the cameras were simply not able to encode images in a rotated manner. If the ISPs were able to generate rotated JPEG files directly, the problem of portrait images would have been solved once and for all in the best possible manner. But we don’t live in that kind of perfect world.
Since doing it the right way was out of the question, the camera manufacturers chose what they thought was the next best thing: Writing a flag into the metadata of the JPEG file that records in which orientation the photo has been taken. The natural choice for this was the Orientation field in the Exif data. Exif is the industry standard for metadata in files generated by digital cameras. It is based on the old TIFF image format, but slightly modified so it can be embedded into JPEG files (among others). The Orientation field itself isn’t an Exif invention either: It was already part of the TIFF specification for years, it simply was not very commonly used. With Exif though, it finally came into the spotlight.
The result of this metadata trickery is that portrait photos are written as JPEG files where the image data itself is stored in the wrong orientation, but a field in a metadata block contains information how the image has to be rotated to restore the proper orientation. What could possibly go wrong…?
Rotation confusion
The main problem with this scheme is that is ambiguous: While applications with full Exif support will show the photos as intended, Applications that just care about the image data and ignore all optional metadata will show the photos in a rotated fashion.
Back when the first cameras with orientation sensors appeared, almost no existing software was able to interpret the Exif Orientation Tag. The situation has improved since, but there are still many programs in existence that don’t care about Exif data.
All kinds of confusion
Even with the existence of the Exif Orientation Tag, transforming the photos so that the image data is physically stored in the proper orientation is still required to resolve any ambiguities. But how?
Using an application that does Exif-based rotation itself, correcting an image is not possible at all, because, well, it’s already shown in the correct orientation, so all bets are off about what happens when doing additional transforms.
This is why many Exif-capable applications offer an option to ignore the Orientation Tag when displaying images. This causes them to behave like old pre-Exif image viewers and allows for manual rotation into the proper orientation – at least if the application also modifies the Orientation Tag appropriately, so that the consistency between the layout of the image data and the Tag is kept.
If an application fails to update the Orientation Tag when rotating an image and instead preserves the Exif information wholesale (including the old value of the Orientation Tag), another zombie-like abomination is generated: Such a file will be displayed properly by non-Exif applications, but it will be rotated wrongly in Exif-compliant applications.
(Apparent) solutions
The good news is that the vast majority of image editing and managemant applications you may come across on desktop systems today is capable of handling the Orientation Tag in a fully compliant and consistent manner.
Transformation of the image data so that the Orientation Tag doesn’t call for further rotation on display is still a desirable process, though – after all, you may still come across software that doesn’t interpret Exif information even today, be it web browsers or photo viewers integrated in home electronics (TVs, DVD/BD players, …).
Options to batch-process whole folders of images to rotate them appropriately are abundant: Some image management tools can do it, there are multiple Windows Explorer add-ons that integrate into folder context menus and finally, there’s exifautotran and jhead for Unix-like OSes.
So everything’s fine again, you think? Not completely. There’s still one thing that poses problems.
Thumbnails and bugs
All the operations mentioned so far only deal with the main image. However, an Exif JPEG file can (and often does) also contain a smaller image – a thumbnail. This is used by some (but not all) image management tools in their picture browsers.
Not all applications rotate the thumbnail along with the main image; some programs just leave the thumbnail as-is, thus making it inconsistent with the main image (exifautotran is an example for that). This results in thumbnails being displayed with the wrong orientation in picture browsers, even though the image is displayed just fine when viewed normally.
OK, you may now think that the thumbnail issue is a minor one, or even not an issue at all if you don’t care about the thumbnails. Again, you would be right if this was a perfect world. In reality though, there’s still room for screwing things up. Let me introduce my new friend, the faulty Exif parser (seen on my Samsung TV, for example).
The thumbnails are encoded within the Exif data, and to make things more complicated (or consistent, if you want to see it that way), the thumbnail also has an Orientation Tag. As a result, an application that parses all the Exif data it can get (regardless of which image it belongs to) will read two values for the Orientation Tag: one for the image itself, one for the thumbnail. If these two values don’t agree because rotation of the thumbnail isn’t consistent with the main image, you may just be unlucky enough that the thumbnail’s value »wins« and you end up with a erroneously rotated image even though you took care to prevent that exact situation from happening. In other words, welcome to the inner circles of hell.
Conclusion
To sum everything up: The Exif Orientation Tag is a mess and the world would be a better place without it. There are so many ways where subtle brokenness of one piece of software can screw up everything that it’s just not funny anymore.
Why can’t cameras not just generate properly rotated JPEG data in the first place? Why do they still rely on the fragile tag hackery? Current cameras have so sophisticated ISPs (with motion tracking, HDR, panorama generation and more funky stuff) that rotation is likely to be no problem any longer. But still I’ve yet to come across a camera doing this. This makes me sad, very sad.
Hehe.
Instead of fixing the issue at the source (the camera), my bet is that “next gen” image viewers will rotate the image based on an extensive, cloud-based analysis of the pictures’ content – completely disregarding any EXIF-tags in the process.
This will be AWESOME.
;)
Windows 8 seems to correctly display the EXIF Orientation tag. There is hope!
10×100 % agree!!!
Helas… Windows 8 indeed corrects the orientation of pictures, but it does not correct the Exif-data (of course). Consequence is that portrait pictures showed on a Samsung smart-tv (via Samsung Allshare, see http://www.samsung.com/global/allshare/pcsw/), will NOT be shown in the correct orientation. Photo-editors (and, of course, Windows itself) show them correctly, so it’s VERY confusing to adapt things in the editor or Windows Explorer.
The best option is to copy the pixels from the jpeg, and rotate them according to the exif data and store it as a new image and delete the original one
Sg: If by »copy the pixels, rotate them and store them as a new image« you mean a lossless JPEG rotation, I agree.
In general, this post isn’t a complaint that a proper solution doesn’t exist, because that’s simply not the case: Running
jhead -autorotworks just fine, so I recommend this to anyone. I was rather complaining about the clusterf*ck of problems that arise simply because the camera manufacturers are too lazy to store properly rotated JPEG files in the first place.It is a mess. Great article, KeyJ! I think what makes this particularly infuriating is that just about everyone uses a modern camera (i.e. iPhone, Android or iPad) to take vast amounts of photos. It’s not a minority-of-devices issue anymore. It’s likely a double digit percentage of cameras out there. Ack!
*Looks awed* Thank you for that, it has thrown some light on the behaviour of my W8 PC.
I use it to view pictures, deciding which ones to incorporate in reports. And, historically, have rotated the picture to the orientation I wanted (which is not always upright – if I’m taking a reference picture of something on its side or upside down).
So when I found they autorotated in W8 I thought “that’s clever”. Then found that when you drag it into word, the orientation is ignored – so you have to rotate it in word anyway. Worse than that, if I want to reduce the apparent height, I have to reduce the width of the rotated image.
What fun.
P.S. The sun and moon are but images stuck to the firmament, and therefore the same size. And orientation :D
Why can’t cameras not just generate properly rotated JPEG data in the first place?
==========> Because Many companies attach higher importance on the speed of
picture capturing, and they haven’t realized that Image Rotation is a big issue.
They may think they would lose competence if they do “Pixels copy, rotation, storing”. Because it really takes much time.
Thanks a lot for this post and the suggestion to use jhead. This turns out to be an excellent solution to a peculiar problem with Apple’s iPhoto. iPhoto automatically makes two copies of photos with portrait orientation, quickly inflating the size of the photo library unnecessarily.
> Sg (2013-05-20 06:20)
> The best option is to copy the pixels from the jpeg, and rotate them
> according to the exif data and store it as a new image and delete
> the original one
The freeware XnView displays photos according to EXIF just fine, and you can fix the photo for other people too by using Tools > JPEG Lossless Transformations > Rotate based on EXIF values.
This will appear to have no effect, but it will actually do the rotation on the image data and reset the EXIF orientation back to normal.
skuppy: OK, but have you checked whether XnView also rotates the thumbnail? If it doesn’t, you should better not use that tool at all.
Thanks for mentioning jhead, it works great on Windows too. :)
Just download the binaries from:
http://www.sentex.net/~mwandel/jhead/
http://jpegclub.org/jpegtran/
The following command seems to be what I have been looking for for years:
jhead -ft -autorot **
I recently published an android application to help with this problem by allowing lossless jpg rotation according to the exif orientation flag: https://play.google.com/store/apps/details?id=net.chartrand.doug.photorotator
KeyJ: Is any solution for this issue?
Ngoc Hoang: The only thing you can do (and which I’ll recommend to everyone) is to
jhead -autorotall images right after importing them from your camera, before even reviewing the images. For Windows, there’s a nice lightweight Explorer integration tool for this: http://savolai.net/software/JPEG-EXIF_autorotate.Doug Chartrand: Since your app is not free. I cannot test your app. Would you mind sharing how you can do lossless JPEG rotation on Android? What is the performance? How long does it take to rotate a jpeg file taken from camera?
It’s worth noting that jhead DOES use the -trim parameter when calling jpegtran, so if the operation is not clean, it CANNOT be reversed.
You’re going too far blaming problems relating with exif thumbnails on the exif orientation tag, as in this case is really the thumbnails that are the problem.
If there was no exif orientation tag, you would still have a problem if the jpeg was rotated without updating the thumbnail. For that matter, a jpeg can be cropped or edited in any other way without updating the thumbnail, which is a much more serious (privacy/security) problem.
Totally, totally agree. But here’s a point many people are missing: why should the EXIF orientation flag be the last word on how a picture should be shown? Sure, for your standard holiday snaps it’ll be right most of the time, but suppose I take a shot of a label stuck with the wrong orientation? Or something lain on its side?
It’s incredibly presumptuous to assume that the device knows which way an image should be shown.
With Windows 10 now forcing auto-rotation of thumbnails on users, this has because a full-blown calamity.
After testing many plugins for a solution for this issue on WordPress.org installations, I’ve confirmed that Fix Image Rotation plugin does work with 4.7.2+ version of WordPress (vanilla).
Thank you for the information. I was having trouble on my website using Perl/Imagick to get the vertical/portrait images to upload correctly, after the exif tag was introduced, although I still have the same digital camera, the images do not upload as they were before. I make a test, Orientation Test with 8 different picture modes and it seems to be working now. Thanks for the info!
I think the answer is performance. You want your càmera ready for next shoot as quick as possible.
I wrote a free open source windows explorer shell extension that auto rotates JPEGs with EXIF orientations. I find it useful to auto rotate my images before showing them in a slide show on a TV or digital picture frame – those devices notoriously don’t respect EXIF rotation.
https://github.com/chrdavis/JPEGAutoRotator
Concerning the rotation of thumbnails I noticed that some apps don’t honor the “orientation” tag when set on the EXIF structure holding the thumbnail, and also they don’t apply the orientation of the main picture to the thumbnail. That’s the case with windows photo import wizard for example.
So to build the thumbnail, one had to get the bitmap of the original picture, resize it to smaller size and then rotate it. That resulting bitmap is the thumbnail that is stored and no orientation tag is set in the exif structure of the thumbnail. That’s what I had to do to make the thumbnails look correct while importing photos in Windows.
The fun starts when neither the bitmap nor the EXIF-oriented version has a proper intended orientation. This happens on some modern smartphone cameras when taking a picture from for example a web application.