Difference in images when using EXR and PNG as output filename

[seb-hoch]

Hello,

first I would like to thank you for the good work and the book, I really like your idea of sharing the wisdom in theory and code!

I noticed a problem when rendering a simulated scene with a camera imaging a calibration pattern. The scene uses a realistic camera with a lens file, a projection Lightsource and a bilinearmesh shape with a calibration texture. If I choose PNG as the output format instead of EXR, the rendered image shows artefacts at the edge of the scene as well as in the middle. The circles appear larger and more fuzzy in the PNG output image. My first guess would be, that this could be due to the quantisation of individual samples on the film. My expected behaviour of pbrt would assume that it calculates internally with floats and when it outputs the PNG file it should quantize the film to uint. So the output file should not influence the image quality in this way. In the images I appended in the bottom, I converted the EXR to PNG by myself and I did not get the artefacts.

Unfortunately I cannot provide the full config file because I'm using a propriertary lens design file. I appended the config file and the images of the same scene rendered with output name ending in EXR and PNG.

Thanks for your help and keep up the good work!

EXR output converted to PNG (center cutout):

PNG output (center cutout):

Config File: `LookAt 0.260 -0.087 0 0 0.845 0 0 0 1

Camera "realistic" "float aperturediameter" [4.836] "float focusdistance" [1.068] "string lensfile" ["lensfile.dat"]

Film "rgb" "float diagonal" [15.78366] "string filename" ["output.exr"] "integer xresolution" [5472] "integer yresolution" [3648]

Sampler "zsobol" "integer pixelsamples"[128]

Integrator "volpath" "integer maxdepth" [5]

WorldBegin

AttributeBegin LookAt 0 0 0 0 0.845 0 0 0 1 LightSource "projection" "string filename" ["full_hd_unicolor_image.png"] "float scale" [50] "float fov" [30] AttributeEnd

AttributeBegin Texture "points" "spectrum" "imagemap" "float scale" [1] "string filename" ["16k_x_16k_calibration_pattern.png"] "float uscale" [1] "float vscale" [1] "string mapping" ["uv"] Translate 0. 0.845 0 Rotate 180 0 0 1 Material "diffuse" "texture reflectance" "points" Shape "bilinearmesh" "point3 P" [-0.4 0 -0.4 0.4 0 -0.4 -0.4 0 0.4 0.4 0 0.4] "point2 uv" [0 0 1 0 0 1 1 1] AttributeEnd

`

[BlueBudgieNeedsWater]

Looks like the pbrt png output has gamma on it and the exr->png does not because the exr is linear. If you have tev viewer installed, you can load the exr and move the gamma slider to see the effect. Higher gamma should make the black-white gradients steeper. This in turns nibbles off a few pixels from the black circle edges.

[seb-hoch]

Looks like the pbrt png output has gamma on it and the exr->png does not because the exr is linear. If you have tev viewer installed, you can load the exr and move the gamma slider to see the effect. Higher gamma should make the black-white gradients steeper. This in turns nibbles off a few pixels from the black circle edges.

Ok, that explains it. Is there an option to control the gamma of the png output file? I want to keep the gradients to measure the MTF in the rendered image.

[mmp]

Nice diagnosis, @BlueBudgieNeedsWater!

There is no option to control the gamma of the PNG file; it is always encoded using the sRGB mapping (see e.g. http://entropymine.com/imageworsener/srgbformula/). This is generally a good thing in that it's a more perceptually uniform allocation of the limited 8-bits per channel.. It wouldn't be too hard to modify pbrt to not do that--see Image::WritePNG in src/pbrt/util/image.cpp. It calls QuantizePixelsToU256 when it's writing a floating-point image out as a PNG; you could either eliminate that call and patch things up or remove the LinearToSRGB8 call in QuantizePixelsToU256.

(Or you might update your output pipeline to handle EXR and then you don't have to worry about any issues from the limited precision in PNGs...)

[seb-hoch]

Thanks for the quick responses! (especially @BlueBudgieNeedsWater ) I just wanted to make sure that there is not something strange going on... I didn't think of a gamma correction because it looked so strange in the edges. @mmp Yeah working with EXR is my current step and there I noticed the issue. Still I haven't found a good way to read in EXR files in Python and use the "Deep" image metadata. I'm just starting with pbrt and currently reading the fourth edition of your book.

[BlueBudgieNeedsWater]

You need to polish a few more things for a lens system MTF measurement. Essentially everything that messes with the black/white gradients has got to go. You do not want any gamma, png compression, noise from the rendering or denoiser acting on the image. Pump up spp until no noise it left. GPU acceleration works really well, so high spp is no problem.

The other issue with MTF is that there is no optical dispersion in the pbrt real lens system, so you won't get a white light MTF. Can be integrated by yourself though.

[seb-hoch]

@BlueBudgieNeedsWater I always thought PNG has losless compression? The noise in the render is probably not an issue, because in the "real" setup there is shot noise from the camera as well. The first results from MTF measurements in the render resembles the real measurements pretty well so I'm happy with the results so far. If there is a problem I will have to go the approach of pumping up the spp and handle the noise by simulating the sensor characteristics, but I don't want to go down that rabbit hole...

Is there a denoiser acting on the image? I wasn't aware of that...

I'm not really interested in the full MTF because I get that from the optical design software, I'm more interested how the MTF measurement is affected in a three dimensional complex scenery. Also there is no need for a polychromatic MTF as I'm working with narrowband filters, but can you elaborate why there is no dispersion in the pbrt real lens system? Is it because the refractive index is not wavelength dependant?

[BlueBudgieNeedsWater]

denoiser is not acting by default on the renders from what I checked, but the imgtool provides the Optix denoiser as an optional step. When I checked the San Miguel scene with the real lens, it was producing quite some fireflies. Or at least that was my impression. So I denoised it. I think that would really influence the MTF. Weirdly enough, I did not seem to get so many fireflies with the perspective pinhole camera. But I am just diving in.

wrt to png8 I was refering to bit depth reduction compared to exr. The gradients will get more step-like, but looking how they are rather step-like anyways and not spread out, the bit depth loss probably won't be noticable in the MTF. I will just keep exr 32bit for as long as I can, Actually, would be a nice test to see the bit depth effect on MTF. Really smooth gaussian edge gives smooth gaussian FFT. More step like produces more sinc-ish FFT.

wrt to polychromatic MTF, from what I gather the lens system prescription only allows constant refractive indices. But this is definitely doable because from the looks of it, the pbrt materials that come with the render actually include optical dispersion n(lambda)

[mmp]

wrt to polychromatic MTF, from what I gather the lens system prescription only allows constant refractive indices. But this is definitely doable because from the looks of it, the pbrt materials that come with the render actually include optical dispersion n(lambda)

Indeed! Dispersion in the RealisticCamera was something that was right on the edge of making it in there but was cut in the end to make the deadline. I'm happy to give some pointers about what it would take to add that. It'd probably be a few tens of lines of code in the end.

[BlueBudgieNeedsWater]

No worries Matt. This is on the users end to include. I am the same guy who asked for it in the pbrtv3 issues. I just do not find that github account anymore, I will attempt it soonTM. It is a lot more productive now to work with pbrt and the gpu acceleration.

[seb-hoch]

@mmp Yeah, I saw all the optical materials included in the source files and thought that there was not missing much code to include that in the tracing of the sensor rays. I'll probably give it a try to implement it by myself and also for the projection LightSource. I was also a little bit disappointed that the realistic camera chapter was not included in the printed edition of the book.

[BlueBudgieNeedsWater]

seb, the realistic camera is described in detail in the v3 edition. I think that's why it would be redundant in v4 https://pbr-book.org/3ed-2018/Camera_Models/Realistic_Cameras

[mmp]

Yep. In the end there were almost no changes to that section, so given the page constraints and given that the 3rd edition is online, we ended up cutting it from the print edition this time.

[seb-hoch]

wrt to png8 I was refering to bit depth reduction compared to exr. The gradients will get more step-like, but looking how they are rather step-like anyways and not spread out, the bit depth loss probably won't be noticable in the MTF. I will just keep exr 32bit for as long as I can, Actually, would be a nice test to see the bit depth effect on MTF. Really smooth gaussian edge gives smooth gaussian FFT. More step like produces more sinc-ish FFT.

Normally I would love to have the bit depth to measure the MTF in a real setup but most of the time you're limited by 8-bit by the sensor and one has to write algorithms to reduce noise and increase signal.

seb, the realistic camera is described in detail in the v3 edition. I think that's why it would be redundant in v4 https://pbr-book.org/3ed-2018/Camera_Models/Realistic_Cameras

Yes I know but I hoped in v4 there would be the dispersion included and maybe more sophisticated lens forms like aspheres or free forms or even polynomial phase screens. Now I have to do all the work by myself :-D

[seb-hoch]

Yep. In the end there were almost no changes to that section, so given the page constraints and given that the 3rd edition is online, we ended up cutting it from the print edition this time.

I totally understand, it was probably a lot of work especially with all the gpgpu changes. BTW would it be very hard to change the gpu backend to the Metal API?

[BlueBudgieNeedsWater]

was playing a bit today and wrt to MTF accuracy, I found out that there is a gaussian filter with default 1.5px applied out of the box. This is going to drag the MTF down. I was able to reduce it to 0.05px, so basically zero and the image got visibly sharper. https://pbrt.org/fileformat-v4#filters

Also the option "disablepixeljitter" elimated all anti aliasing, but lots of jaggies of course.

Anyways, good fun testing things. The gpu acceleration really does a good job. 4070Ti is like 100-200x faster than my mighty Xeon W-2123 :)