search

compphoto / IntrinsicCompositing

Code for the SIGGRAPH Asia 2023 paper "Intrinsic Harmonization for Illumination-Aware Compositing"
https://yaksoy.github.io/intrinsicCompositing/
Other
48 stars 10 forks source link

Harmonized output image using inference.py not the same as result for chair example. #4

Closed Saumya-svm closed 9 months ago

Saumya-svm commented 9 months ago

I have attached the picture for the output I got when running inference on the cone chair example. It differs greatly from the one displayed in your result. For example, the sunlight shade on the chair is not reproduced while running inference, whereas, in the results shared, the chair is adjusted for the sunlight. Please explain what caused this.

Inference Image ori_out

Result in the Paper: ori

CCareaga commented 9 months ago

I think this is a bug introduced while I was cleaning up the code to release. It seems like the albedo harmonization network isn't giving the same output I was getting when I made that figure. I'll try to do some testing and see if I can figure out what's going wrong. I'll let you know when things are working.

Thanks!

CCareaga commented 9 months ago

I've found two discrepancies between the GitHub codebase and the original codebase used for the paper. The first discrepancy was a trivial mask pre-processing error when performing albedo harmonization. After fixing this bug, the result for this example looks like this:

cone_chair

The second discrepancy is actually a bug in the original implementation I used to generate figures for the paper. The original albedo harmonization training and testing code assumed that the shading images were stored as 16-bit values, and normalized them to [0-1] accordingly. But when generating results I was using 8-bit shading images. This meant that the albedo being fed to the network was incorrect (due to the low-contrast shading values). When I prepared the code for release, I fixed this bug without thinking about it meaning the GitHub code does not have this issue. I believe the GitHub code is a more accurate implementation since the albedo harmonization network is receiving the correct albedo as input. In order to maintain reproducibility, I've added a flag to the inference script called --reproduce_paper that will use the logic and weights from the original implementation (10edf3b6253568bfd52ce838ca8c9393f1254bf6). Without this flag, the code will run correctly and use better weights for the reshading network. Here are the results you should see for each setting of this flag:

with --reproduce_paper without --reproduce_paper
cone_chair cone_chair

The same flag has been added the interface. I'll add a note to the README explaining this as well. Thank you for bringing this to my attention!