search

NVlabs / neuralangelo

Official implementation of "Neuralangelo: High-Fidelity Neural Surface Reconstruction" (CVPR 2023)
https://research.nvidia.com/labs/dir/neuralangelo/
Other
4.31k stars 387 forks source link

Error occured when training on the synthetic data that not from nerf #126

Closed Then0109 closed 11 months ago

Then0109 commented 11 months ago

Dear author, Thanks for releasing the codes. After training the model on real scene data,i am trying to train the synthetic data. And i choose a synthetic dataset from other model. However,i don't know if it's the data doesn't fit the model that affects the results.

If i just input the origin rgba images into the model,then the config generation process would be interrupted by the 'bgcolor' error.

If i change the rgba images to rgb images and input the rgb images,the process would successfully finish,while the result is very bad.

In fact,the output rgb image synthesis is white without any information.Here are some of the dataset information and results. Could you please help me with my synthetic data test? Appreciate it very much.

dataset from ref-nerf (train on the linux system): 屏幕截图 2023-09-25 175412 屏幕截图 2023-09-25 175436r_0 results: 屏幕截图 2023-09-25 175326

mli0603 commented 11 months ago

Hi @Then0109

The normal/depth/opacity suggests that the poses of the camera are likely wrong. Usually, for object-centric scenes, you would expect a sphere in the middle of the image during the start of optimization.

Then0109 commented 11 months ago

Thank you @mli0603 for the tip for checking out the camera poses. Actually, I use the colmap to directly predict the poses of camera. I check out the poses of camera with ipynb then,it looks right. image I don't know if it's the 'RGBA' data that is not suitable for the 'right' llff type data input demand,because when i changed the data type to 'nerf_blender',then the error that the AttrDict object has no attribute 'bgcolor' occured. In the end, it's working when i convert the RGBA data into video,follow the whole process of data_preprocessing.md and input to the model. Although the result is not very good to some extent,the PSNR kept in 15-17db in the train process.

Anyway,thank you again sincerely!

mli0603 commented 11 months ago

I see. Good to know you have resolved the problem by using the poses from COLMAP. I think there is a convention difference between LLFF and COLMAP, which may have caused your previous issue. I will close this for now as things are working.