Intensity encoding function changed for Perseverance since Sol 511

[isenberg]

Hi Kevin, since Sol 511 I noticed a different intensity transfer function in use for the image data in the PNGs on https://mars.nasa.gov/mars2020/multimedia/raw-images which most likely need an adaption for the mars-raw-utils in case it isn't a bug in the original PNGs. The change is visible in Mastcam, Navcam, Hazcam.

Example, comparing the sundial calibration target at similar solar time on Sol 510 and 512 with identical clear sky light conditions:

mru m20-fetch -c MCZ_LEFT -s 510 -S CAM03014 -o /tmp
mru m20-fetch -c MCZ_LEFT -s 512 -S CAM03014 -o /tmp
mru calibrate -P m20_zcam_ilt -i /tmp/ZL0_0510_0712219006_707ECM_N0261222ZCAM03014_048085J01.png
mru calibrate -P m20_zcam_ilt -i /tmp/ZL0_0512_0712391417_394ECM_N0261222ZCAM03014_048085J01.png

Results uploaded: https://areo.info/mars20/files/ZL0_0510_0712219006_707ECM_N0261222ZCAM03014_048085J01-rjcal-ilt.png https://areo.info/mars20/files/ZL0_0512_0712391417_394ECM_N0261222ZCAM03014_048085J01-rjcal-ilt.png

From looking at the histograms in the original PNGs I assume the data has been converted to match the sRGB intensity function since Sol 511. Or the inverted camera-LUT1, the squareroot encoding has been applied. There is also a low chance that a different LUT already on the rover is applied, but from the appearance of the histogram I assume sRGB.

What do you think? Can you contact the Perseverance web team to ask them what happened?

Holger

[kmgill]

I did notice this. At first I thought it might be a different sun angle causing blooming in the optics, but now I'm not so sure. I'll ask around and see what's up.

[isenberg]

It's definitely a different encoding used now. I noticed it first on the Navcam full frame images on my own automated website https://areo.info/mars20 as those showed completely different colors due to the stronger effect with my color processing and non-linear intensity change to improve contrast. Then checked out the details and also Hazcam and Mastcam are all different starting with Sol 511. WATSON doesn't appear to be affected, but I haven't checked all details there yet.

[kmgill]

Looks like there was a pipeline update around the same time as when this started which may be the culprit. I don't presently have access to the EDRs, so I'm not sure yet what the issue/change was.

[isenberg]

Any news on this? Is it planned to stay that way? There is also a chance that the new web PNGs are the original Earth-received 8 bit files (ECM in the PDS). That would even increase the quality after decoding to 12 or 16 bit. But there is also a chance that by error an sRGB intensity function was applied as both are quite similar, though not identical.

[kmgill]

I figure it's a difference in how they're computing the values needed to convert from the sensor values to the 8 bit PNG range.

The images have been posted in their 8 bit EDR format since the beginning. This is before any LUT conversion back to 12 bit (within a 16 bit buffer) or other calibration. Non-mission members don't generally see the calibrated stuff until it hits the PDS archive.

[isenberg]

Thanks for the update! Good to hear that it was the Earth-received 8 bit data before. Most likely for the web-published PNGs then in addition linearly stretched to full intensity, but that's easy to compensate. The current format has the advantage not needing to guess the used 12-to-8 bit LUT of the 32 possible on the camera, but has the disadvantage that the quality (dynamic range) is reduced as the web PNGs are only 8 bit and the real EDR data 12 bit. As for most images after the first few sols LUT#2 is selected, the LUT-guessing problem doesn't really exist in practice.

If they switch the LUT in the next days for some time, then it will be an interesting experiment actually to keep it this way for some days, as it may be the case that some other LUT surprisingly increases the quality. I saw some hints for that in the first 2 Sol weeks. That's counterintuitive, but if you look at the images it's visible.

[isenberg]

Have you heard any news on this? What was on the processing level changed? From your previous message I interpret it that now the fully companded images are shown but linearly scaled down from 12 to 8 bit? Fully-companded = 12 to 8 bit applied on rover and 8 to 12 bit applied after receiving on Earth.

But if that's the case: Why is the result still different when comparing it to previous sols? I only get a similar result when applying the sRGB gamma function and then the 8 to 12 iLUT.

[isenberg]

With the new PDS raw data release of today I can confirm my previous assumption about the changes in the web-PNG production pipeline. The sRGB intensity curve is indeed now applied, that means gamma 2.2. I could confirm that by comparing the web-PNGs and raw camera JPGs (EJP files in the PDS) for the following 2 Navcam images from sol 508 and 511: https://areo.info/mars20/ecams/0508/tn/NLF_0508_0712035900_230ECM_N0261222NCAM00501_01_295J_calib01_areo.info.jpg.html https://areo.info/mars20/ecams/0511/tn/NLF_0511_0712320966_273ECM_N0261222NCAM00501_01_295J_calib01_areo.info.jpg.html

Results from my analysis about the JPL production pipeline for the web-PNGs on https://mars.nasa.gov/mars2020/multimedia/raw-images :

• Sol 510 and before: read raw Navcam JPG (EJP data), cut off lowest 10% and highest 90% intensity, write PNG
• Sol 511 and later: read raw Navcam JPG (EJP data), apply sRGB intensity curve (gamma 2.2), then cut off lowest 10% and highest 90% intensity, write PNG

I suspect just a bug in their processing pipeline. That can easily happen when reading linear RGB JPGs, like the Navcam-produced JPEGs on the rover are, with the imagemagick convert tool or also Python image loaders. For imagemagick that can be avoided by using for example convert INPUT -set colorspace RGB -colorspace RGB OUTPUT. That overwrites the default assumptions of sRGB and sets fixed to linear RGB.

[kmgill]

OK, thank you. I just added some additional steps in the M20 navcam calibration to deal with it. Also adds a pair of calibration profiles that'll need to be copied into ~/.marsdata

[kmgill]

Resolved in M20 pipeline