Image color shift in DC

[apinkney0696]

Last Friday I noticed a difference in the saturation/color between my images on my desktop image viewer and the online DC image viewer (when viewing side by side on the same monitor). I checked on a couple of different computers, and it does seem like the images become desaturated or color corrected after being ingested into the DC. This is an issue of course because we want the portrayed color to be accurate to the physical items. This ticket is just a placeholder to do a little investigation about what might be going on here - the specific images I'm talking about are in M:/Rare Books/ MS 19 and online here: https://digital.sciencehistory.org/works/gygpu42

Thanks!

[jrochkind]

I ran into a similar issue in my PDF work. I don't entirely understand it, but I'm starting to understand some of the issues.

the basic issues are when switching from one format to another, like TIFF to JPG -- which needs to be done for display on the web as you can't display TIFF. There can be issues with "color profiles" -- there may or may not need to be a translation, which the software we are using may or may not be doing correctly.

One thing that has already come up for me is that we may want to understand better our original digitization process, and the choices made there, and if it might make sense to make different choices to make it easier and/or possible to do format conversions without having to change color profiles -- like there may be a choice of how/what color profiles to use in our original TIFFs, that could be a different choice with different consequences.

I am only starting to understand these issues.

[jrochkind]

I would be curious exactly which files you are seeing/detecting color differences between among:

1. Your original files on the M drive
2. The TIFF downloaded from DC (should be identical to your original files on M drive, or else something is startling!)
3. a Full-Size JPG downloaded from DC -- does it match 1 and 2, or not?
4. The online pan-and-zoom viewer -- is this the only one that is off?

[apinkney0696]

Eddie mentioned to me that I should specify that the issue seems to only be affecting the online derivatives, and we should rule out any harm to the originals.

Thanks, Jonathan - this is the first time I noticed a difference between my photos and the derivatives. I believe this is the first time I uploaded photos I took using an updated version of the imaging software too, so perhaps there is an issue with the color profiles related to the update. I am not in the office today, but on Wednesday and Thursday this week I am planning on working through more object photography. I will do some investigation to the color profiles and send you what I find.

I did not do any downloads!! I should have tested that. I only noticed differences between my M drive originals and the online viewer/thumbnail display.

[apinkney0696]

Okay, so I just downloaded both the TIFF and JPG versions, and they are definitely more saturated than the online thumbnails and pan-and-zoom viewer versions. I haven't compared those to the originals in M, but I would venture to guess they're identical based on the coloring.

It seems that 1. 2. and 3. above are identical, and 4 is the only thing off.

[jrochkind]

OK, I have some ideas, although some might require new releases/features from some software we use, which is tricky.

[jrochkind]

Documenting my current hypothesis, before leaving this for a while....

Based on understanding: Colors look as expected, without problems, in downloaded JPG and TIFF. Colors look problematic in on screen thumbnails, and online pan/zoom viewer.

First, context: All images are in a certain "color space", and different images even of the same image format may be in different color spaces. Some images, in image formats that support it, also have embedded "image profiles", which are basically embedded color space specifications.

For our thumbnails, we are converting them to sRGB color space -- this is based on advice that this makes sense for browser thumbnails, to make them smaller file size, and for most universal viewabilty. Advice:

• https://developers.google.com/speed/docs/insights/OptimizeImages
• http://libvips.blogspot.com/2013/11/tips-and-tricks-for-vipsthumbnail.html
• https://github.com/jcupitt/libvips/issues/775

We do not do this for our JPGs prepared for download. So my guess is that this process of converting to sRGB is corrupting the colors.

• I am not sure if color change like this is expected in a conversion from sRGB.
  • It could be that the conversion is not being done properly -- either because we are misconfiguring our tools, or because of bugs in tools.
  • Or it could be that such a conversion necessarily and expectedly will shift colors in some cases.

We could stop doing this color space conversion when we create our thumbnail JPGs. At the cost of maybe some increase in thumbnail size, but it's probably minimal. If we see that it solves our color issue, it is probably the way to go.

For the online zoom-and-pan viewer, we are using vips dzsave to create tile JPGs. It could likely be that this software is also doing the color space conversion.

This is harder to fix, because I don't believe there is currently a flag to ask it not to. We'd have to file an issue with vips, and wait for such a feature to be in a vips release. But it can be a challenge to install the latest vips on heroku -- we rely on it being available in the ubuntu package repository for the version of ubuntu we are using on heroku, with no good way to install other versions of vips.

So it might be a challenge to get a fix deployed soon for the viewer, but if this is what we want, we should report to vips anyway.

[apinkney0696]

When I exported images in sRGB, they showed no color shift in their thumbnails or the image viewer. I think this supports your hunch that it's the process of converting to sRGB corrupting the colors. Higher FADGI levels require Adobe RGB, so I like your suggested solution to stop the color space conversion, despite an increase in thumbnail size. Sounds like we should also put a ticket into vips and hope for the best.

[jrochkind]

Thanks for this!

Based on this, my theory, described technically is actually: When creating DZI tiles for the viewer, It is not converting the (Adobe RGB) colorspace. But it IS writing to JPG tiles that have metadata saying they are SRGB when in fact they are still Adobe RGB, leading to corruption in display of the colors.

This is something we'll have to report/discuss with vips.

[jrochkind]

Although the workaround is having our software actually convert to SRGB before creating tiles... if the conversion is actually done appropriately, this should be invisible and colors should look proper. This is what I learned from working on this same issue with JP2 and PDF -- if we actually convert from Adobe RGB to SRGB (for the derivatives only the original remains Adobe RGB), there is no visible color change on our monitors. The problem comes from the color profile metadata being wrong without a conversion.

[apinkney0696]

Got it. Thank you for the explanation!

[jrochkind]

download derivatives do not exhibit problem -- both viewer AND on-screen thumbnails DO. Annabel confirms.