Issue with Jpeg dataset even with correct weights (hdrcnn_params_compr.npz)

[ittim4]

Downloaded 96 image jpg dataset from http://hdrv.org/hdrcnn/ , section "Testset Reconstructions" i.e. link (http://hdrv.org/hdrcnn/material/sga17_testset.zip). 1)For testing, used jpg images in "testset\jpg\input", but these already seems darker. Is it correct folder to test with jpg datset ? Why all input jpg images already seem darker?

2) When run for "testset\jpg\input\img_0001.jpg" with hdrcnn_params_compr.npz; in "out" folder I see 3 files: 000001_in.png, 000001_out.png and 000001_out.exr. Here, "000001_in.png" is much darker than original "testset\jpg\input\img_0001.jpg" which is probably expected, as I read your comment from other thread that these in and out jpg images are still LDR with "3 stops" reduced. However, I see "000001_out.png" almost same as "000001_in.png" and no visible difference at all and both are too dark. When debugged/looked in further, it seems as per script run log, there are NO saturated pixels detected! Copying script run log for reference below. Can you comment why it says (Saturation: 0.00%) ?? Btw - I checked other testimages out of 96 jpg images, behavior is similar for all! With png dataset, I could see saturation pixels were detected i.e. it was not 0%, but it seems to be the case when I run with jpg dataset! Can you please help explain this ? Request to suggest how to test on jpg dataset downloaded from above mentioned link?

=========== sample run log:

hdrcnn-master>python hdrcnn_predict.py --params hdrcnn_params_compr.npz --im_dir D:\HDR_ML\Dataset\data2_jpg_hdrcnn\testset\jpg\input\img_0001.jpg --width 1024 --height 768 ←[95m

←[0m←[95m HDR image reconstruction from a single exposure using deep CNNs

←[0m←[95m Prediction settings ←[0m←[95m ------------------- ←[0m←[95m Input image directory/file: D:\HDR_ML\Dataset\data2_jpg_hdrcnn\testset\jpg\input\img_0001.jpg ←[0m←[95m Output directory: out ←[0m←[95m CNN weights: hdrcnn_params_compr.npz ←[0m←[95m Prediction resolution: 1024x768 pixels ←[0m←[95m -------------------------------------------------------------------

←[0mNetwork setup: ←[0mWARNING:tensorflow:From C:\anaconda3\envs\tf\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.fram ework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. 2019-07-02 15:18:58.592733: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2 2019-07-02 15:18:58.600972: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 4. Tune using interop parallelism_threads for best performance.

Loading trained parameters from 'hdrcnn_params_compr.npz'...←[0m done ←[0m Starting prediction...

←[0mFrame 0: 'D:\HDR_ML\Dataset\data2_jpg_hdrcnn\testset\jpg\input\img_0001.jpg' Reading...←[0m done←[0m←[95m (Saturation: 0.00%) ←[0m Inference...←[0m done ←[0m Writing...←[0m done ←[0mDone! ←[0m

[gabrieleilertsen]

The problem here is that the jpg images have been reduced in exposure, in order to illustrate differences between input LDR image and reconstructed HDR, e.g.:

http://hdrv.org/hdrcnn/material/testset/html/img_0001.html

Thus, there are no pixels close to 1 and considered saturated, and the inference script does nothing to the image. In order to actually use the images for running inference, you have to convert the HDR images to LDR. For example, in Matlab you could do:

I = exrread('exr/ground_truth/img_0001.exr');
imwrite(I.^0.5, 'ldr_image_001.png');

Here, I used a gamma correction, I.^0.5, in order to convert the linear HDR pixels to LDR. You can also use a camera curve, such as the sigmoid function that we used in the paper.

[ittim4]

Thanks a lot gabrieleilertsen, that explains why jpg/input images were dark to begin with. Am still new to exr format and trying to understand few details before getting into gamma correction. a) In exr linear fp16 format (i.e. before gamma correction), my understanding is pix values represent values as scene referred and not display referred..is it correct? Also, whats relationship between values in exr to "nits" ; trying to understand what would be max nits present in "predicted or final reconstructed" hdr image output here? For e.g. is it fair to assume value 1.0 represents ~100 nits as 1.0 should ideally represent max nits of SDR ? b) Thanks again, can use variants of gamma correction to create ldr from exr; btw - which matlab module was used for exr handlng, is it this or can you suggest one?

[gabrieleilertsen]

a) Yes, the LDR images are display-referred and the HDR are scene-referred, i.e. linearly correlated with luminance. Nits is the same thing as cd/m^2 (candela per sq. meter), and I is common e.g. for TV specs. To calibrate the HDR images to cd/m^2, you have to scale the HDR image intensities appropriately, and the scale depends on the scene. In a long-exposure night scene you need smaller scaling than in a short-exposure day scene. However, in most cases you don't really have to be concerned about physical calibration. For a rough guideline on luminance values, you can look here.

b) Yes, for EXR you need the OpenEXR library. This is the official GitHub repo I believe.

[ittim4]

Thank you gabrieleilertsen for comments on a, b! Aware of nits or cd/m^2; was trying to understand say values like 0.5, 1.2 or 3.4 in exr file will translate to what 'nits' in actual content (i.e. in the context of 'scene referred' and not 'display referred'), but I believe- that depends on luma scale factor (at capture time) which may not be known here. let me know if there is a way to interpret nits (scene referred) at pixel(x,y) from exr values at pixel(x,y) or maybe that's not possible here. Thanks again for clarifying earlier questions. (P.S: sorry to respond late, was away for a week).