search

NVlabs / nvdiffrec

Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".
Other
2.13k stars 223 forks source link

Is there any way to use datasets from instant ngp? #26

Closed jpbalarini closed 10 months ago

jpbalarini commented 2 years ago

Hi, I am trying to train nvfdiffrec straigth from instant-ngp NERF datasets but I'm blocked. The instant-ngp dataset uses images without alpha and it seems like nvdiffrec needs images with alpha. Also, nvdiffrec uses a train/test/val dataset, is there a way to use the same input as instant-ngp uses?

Thanks!

jmunkberg commented 2 years ago

Hello @jpbalarini

Yes, we do require alpha. The foreground segmentation mask is essential for our shape optimization to work. See the paper for details.

We should be compatible with the standard synthetic nerf datasets from https://github.com/bmild/nerf, see e.g., https://github.com/NVlabs/nvdiffrec/blob/main/configs/nerf_chair.json for an example config.

What are the differences between the instant-ngp and the nerf synthetic dataset versions? Our nerf dataset reader is here: https://github.com/NVlabs/nvdiffrec/blob/main/dataset/dataset_nerf.py , hopefully it is easy to customize to your needs.