search

nerfstudio-project / viser

Web-based 3D visualization + Python
https://viser.studio/latest
Apache License 2.0
728 stars 45 forks source link

the viewer in nerfstudio shows a rendered image from the 3DGS result at that current perspective，Is this right? #232

Open smart4654154 opened 3 months ago

smart4654154 commented 3 months ago

Thank you very much for your reply（https://github.com/nerfstudio-project/nerfstudio/issues/3090） I am currently studying NERF and 3DGS, and I have some new question

1.What does "at that current perspective" mean, and how does nerfstudio obtain current perspective ? 2.the viewer shows a rendered image from the 3DGS result at that current perspective，Is this right? thank you

brentyi commented 3 months ago

Hi!

  1. "At the current perspective" means using a 3D camera defined by the web-based viewer. We use something similar to the "OrbitControls" controller in threejs to interact with the viewing camera pose: https://threejs.org/examples/misc_controls_orbit.html
  2. Yes, cameras for NeRFs and 3DGS are handled identically in the context of the viewer!
smart4654154 commented 3 months ago

Thank you very much for your reply I would like to know more about this content. Can you give me some suggestions? When I rotate in the viewer, the Python code will get the current perspective. The Python code uses the perspective information to re-render the image and pass it to the viewer. I would like to know the Python code for this process. Can you give me some suggestions? Thank you very much.

brentyi commented 3 months ago

There's a lot of steps here so it's hard to write a short summary, but you could look at:

Nerfview might also be a useful reference: https://github.com/hangg7/nerfview