Thumbnails sometimes do not honour image rotation and appear sideways

[haku]

not sure why, maybe there are multiple ways to implement rotation and libraries only support some? or maybe phones always save photos the right way up which i why this is not a more prevalent issue?

[girlpunk]

Some phones save the photo the way the sensor is oriented, and then use a JPEG field to indicate it should be displayed rotated. That might be what's causing this?

[haku]

yea that is what that last commit was supposed to add, but seems to not work for all jpeg files yea standards etc my next idea is to add support for reading exif thumbs

[haku]

further debugging to figure out why it is only sometimes working: it depends on which JPEG decoder actually gets used.

• if it decodes using com.sun.imageio.plugins.jpeg.JPEGImageReader (first choise) then exif rotation is ignored since it hard-codes Dimension/ImageOrientation="normal".
• if it fails to decode with the JVM provided decoder[1], it falls back to com.twelvemonkeys.imageio.plugins.jpeg.JPEGImageReader which does populated ImageOrientation with the actual value.

tested with JVM 11, not checked if later JVMs have an updated JPEGImageReader.

note i set this decoder ordering specifically cos the jvm-internal one seems to be quite a bit faster and i did not realise it was incomplete like this.

[1] eg fails with javax.imageio.IIOException: JFIF APP0 must be first marker after SOI

[haku]

nope, still hard-codes no rotation: https://github.com/openjdk/jdk/blob/12028000db2ef3b1c784af119c495aa3ef9590cf/src/java.desktop/share/classes/com/sun/imageio/plugins/jpeg/JPEGMetadata.java#L969

[haku]

next idea was to read thumbs from headers to avoid needing to regenerate and hope those were orientated correctly... except that the com.sun decoder fails to read those also. it seems in only supports JFIF where as the twelvemonkeys also supports JFXX and Exif, and seems most thumbs are in Exif...

sigh. this is coming down to a speed vs completeness trade off... i could set it to always use twelvemonkeys decoder and the problem would go away, but it would be much slower. i should add metrics to measure how much slower...

a possible mitigation is to make use of thumbs in headers, though these are inconsistent in resolution so would need to come up with some selection criteria like if its close to 200x200 then use as is, if much bigger then shrink it down. though either way the thumb would get recompressed since there is no way to read the compressed thumb bytes directly.

[haku]

added a metric: 948ec4ae830e9ced12d0f39c051821210e688627

possibly unsurprisingly, performance very much varies depending on the mix of files being processed without thinking i through all my phone phones from this year at it and it was Slower when using what i had thought of as the "optimised" path because the "try with the faster decoder then fallback" actually takes longer if it Always falls back i should have seen that coming really

maybe just delete this "optimisation" and stop over thinking this? https://github.com/haku/mediatoad/blob/948ec4ae830e9ced12d0f39c051821210e688627/src/main/java/com/vaguehope/dlnatoad/util/ImageResizer.java#L53-L69

[haku]

TODO for another day: read exif thumbs if present and dimensions are usable. possible logic: s = max(width, height)

• if thumb with 100 < s < 300 px: use as is
• if all thumbs have s < 100 px: full generate
• else: resize largest thumb to 200x200 px

or lazier logic would be always use embedded thumb no matter what size it is, not sure about failure modes i have vague memories of photoshop or similar sometimes embedding huge thumbs in tiny images etc i guess i should somehow survey what is typical for embedded thumbs...