search

VITA-Group / GNT

[ICLR 2023] "Is Attention All NeRF Needs?" by Mukund Varma T*, Peihao Wang* , Xuxi Chen, Tianlong Chen, Subhashini Venugopalan, Zhangyang Wang
https://vita-group.github.io/GNT
MIT License
338 stars 24 forks source link

color issue #5

Closed CCOSerika closed 1 year ago

CCOSerika commented 2 years ago

Hi, author. Thanks for your excellent work on GNT again. I tested the code on nerf_synthetic drums model and the result looks good. But I met an issue that, when it comes to an indoor scene, the color of images predicted looks strange. Just as 1 It seems that black and white reversed partly. Where lies the problem?

CCOSerika commented 2 years ago

Perhaps I found the key. It is caused by two lines of code in data_loader here.

rgb = rgb[..., [-1]] * rgb[..., :3] + 1 - rgb[..., [-1]]
 src_rgb = src_rgb[..., [-1]] * src_rgb[..., :3] + 1 - src_rgb[..., [-1]]

I attempted to comment them out, and the color of images became normal, but motion blur appeared. I wonder how they work in the code.

MukundVarmaT commented 2 years ago

Apologies for the delayed response. Nerf Synthetic images contain 4 channels, the last indicating occupancy of an object in the scene (or simply put a mask of the object). Hence I assume they are not relevant to the dataset you are testing it on. Regarding the motion blur, it is tough for me to comment without the data, perhaps you could point me to the dataset you are testing on (and the checkpoint u are attempting to evaluate) and I could try them locally.

CCOSerika commented 2 years ago

Great appreciation for ur kindness. I get it and have solved the issue. Congratulations on such an excellent work. Next I'm going to optimize the rendering time. Hope share with me if u have any ideas.