search

med-air / EndoNeRF

Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery
https://med-air.github.io/EndoNeRF/
172 stars 14 forks source link

3D reconstruction failed on SCARED #15

Open sincerely1 opened 1 year ago

sincerely1 commented 1 year ago

hello, I try to use your code in SCARED dataset. I set the masks to be a black image. I use the ground true camera pose and use https://github.com/ShuweiShao/AF-SfMLearner to get depth image. But the result is failed。。I only iter 5W rounds。But I use your dataset is ok。Do you meet this problem？

yuehaowang commented 1 year ago

Thanks for your question. As you didn't upload a full image, I can partially observe the failed results. I would suggest checking the camera poses and estimated depth. For instance, you can back-project the RGB and depth maps to reconstruct a point cloud frame and see if the point cloud looks good.

sincerely1 commented 1 year ago

Thank you for you answoer. The origin image is frame-000000-depth.png. The point clound like Cloud-Compare-q-Ql-U8-BB0hp.png The result is so bad. I do know why. I will use the STTR-Light try again.

yuehaowang commented 1 year ago

Thanks for providing the results. I guess this problem might relate to camera poses. As mentioned in the readme, camera parameters are organized in the format of LLFF. Since the camera will move around, I may suggest disabling NDC in the training procedure.

sincerely1 commented 1 year ago

Thank for your advice. I will try this again.

zihui-debug commented 1 year ago

@sincerely1 I have the same problem. Have you find the reason?