NVlabs / BundleSDF

[CVPR 2023] BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
https://bundlesdf.github.io/
Other
1.04k stars 114 forks source link

3D reconstruction consultation #126

Closed cynthia-you closed 8 months ago

cynthia-you commented 10 months ago

Hi 温老师~ After try bundleSDF on GTX1066 ti( loftr always failed), i pulled image on rtx3070, and it run successfully (run_video/global_refine/draw_pose). But the generated 3d mesh(textured_mesh.obj) appears to come from the perspective of a particular frame, not the starting or ending frame. I'm wondering how can i get the completed mesh just like the readme.md showed(the drill demo) , thanks~

wenbowen123 commented 10 months ago

The mesh should be complete. Did you rotate the mesh to viz from a different perspective?

cynthia-you commented 10 months ago

Screenshot from 2023-11-30 15-27-56 Screenshot from 2023-11-30 15-28-16

after run 'global_refine' only mesh_cleaned.obj and textured_mesh.obj generated, and i droped the obj in Meshlab.

cynthia-you commented 10 months ago

The mesh should be complete. Did you rotate the mesh to viz from a different perspective?

A, i see, it is becuz every time i run the bundlesdf, the 3d reconsturction results are diffrent: Screenshot from 2023-11-30 16-50-18 Screenshot from 2023-11-30 17-25-54

cynthia-you commented 10 months ago

######################### another Q ########## When i run dataset 'milk' no Pdb showed, but in my customed dataset , i must manually enter 'c' every epoch, can i do sth to revise that? Screenshot from 2023-12-01 09-22-13

Jingranxia commented 10 months ago

######################### another Q ########## When i run dataset 'milk' no Pdb showed, but in my customed dataset , i must manually enter 'c' every epoch, can i do sth to revise that? Screenshot from 2023-12-01 09-22-13

你这里是连续帧出了问题吧,没有匹配上

cynthia-you commented 10 months ago

######################### another Q ########## When i run dataset 'milk' no Pdb showed, but in my customed dataset , i must manually enter 'c' every epoch, can i do sth to revise that? Screenshot from 2023-12-01 09-22-13

你这里是连续帧出了问题吧,没有匹配上

有可能的,但是不管是示例数据集还是自制数据集都频繁发生最后的mesh残缺。

Jingranxia commented 10 months ago

######################### another Q ########## When i run dataset 'milk' no Pdb showed, but in my customed dataset , i must manually enter 'c' every epoch, can i do sth to revise that? ######Screenshot from 2023-12-01 09-22-13

你这里是连续帧出了问题吧,没有匹配上

有可能的,但是不管是示例数据集还是自制数据集都频繁发生最后的mesh残缺。

作者的示例数据集生成的瓶口处是会残缺的,如果别的地方也残缺了,那就是环境可能不对

wenbowen123 commented 10 months ago

A, i see, it is becuz every time i run the bundlesdf, the 3d reconsturction results are diffrent:

This is weird. Overall the mesh should be fairly similar. Another weird thing is your texture seems black. Did you try HO3D data as well?

cynthia-you commented 10 months ago

A, i see, it is becuz every time i run the bundlesdf, the 3d reconsturction results are diffrent:

This is weird. Overall the mesh should be fairly similar. Another weird thing is your texture seems black. Did you try HO3D data as well?

Cuze the meash and material_0.png are incompleted, so the mesh looks black on some view, if i rotate the mesh , the missing parts will be more visible.
And in the process of neural field training, the error is very big, the error is up to 1X on rgb_loss and sdf_loss, so the model is incomplete(my assumption). I havent try HO3D yet.

wenbowen123 commented 10 months ago

in many sense, this is abnormal, though I do not have any clues. Did you make any local changes to the code? I'd recommend to start from scratch (cloning the repo, restart docker, etc).

52THANOS commented 9 months ago

######################### another Q ########## When i run dataset 'milk' no Pdb showed, but in my customed dataset , i must manually enter 'c' every epoch, can i do sth to revise that? Screenshot from 2023-12-01 09-22-13

你这里是连续帧出了问题吧,没有匹配上

有可能的,但是不管是示例数据集还是自制数据集都频繁发生最后的mesh残缺。

######################### another Q ########## When i run dataset 'milk' no Pdb showed, but in my customed dataset , i must manually enter 'c' every epoch, can i do sth to revise that? Screenshot from 2023-12-01 09-22-13

你这里是连续帧出了问题吧,没有匹配上

有可能的,但是不管是示例数据集还是自制数据集都频繁发生最后的mesh残缺。

可以私下交流一下吗,我有些问题