search

19reborn / NeuS2

[ICCV 2023] Official code for NeuS2
Other
597 stars 38 forks source link

Reconstructing a dynamic scene with fast movement #71

Open albb20 opened 2 months ago

albb20 commented 2 months ago

Hi, as the title says, I'm trying to reconstruct a dynamic scene with three views with large movements of a rigid body. At the moment, the rendered mesh shrinks every frame until it disappears totally after 3-4 frames. See screenshot below. grafik

My idea was that it must has something to do with the predict global move step, since i have large movements between the frames, as shown below. Do you have an idea, how to change the globalmove parameters for such a scene? grafik

Strangely, the reconstructed models are not even close to the original images. If i put in the coordinates and images to NGP, the shape of the model is fine.

Thanks in advance.

19reborn commented 2 months ago

Hi @albb20, for such case, you may need to increase the training epoch for global movement optimization with less learning rate. Also, increase the aabb_scale or decrease the scale may also be helpful.

albb20 commented 2 months ago

Hi, thanks for your answer. I'm working in the NeuSPP branch at the moment, where i think aabb must be 1. But since i don't need unbounded scenes, switching to main branch should be beneficial for me. For the current data, i already tried playing around with global movement. The results above were obtained with learning rates 5e-5 and more epochs. I will try further.

albb20 commented 3 weeks ago

Another question: Is there a quick way to remove the possibility to deform the mesh between the timesteps, i.e. to allow only for rigid body reconstruction in a dynamic scene?