search

zju3dv / NeuralRecon

Code for "NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video", CVPR 2021 oral
https://zju3dv.github.io/neuralrecon/
Apache License 2.0
2.07k stars 299 forks source link

Inference with other well known dataset #23

Closed kimsj0302 closed 3 years ago

kimsj0302 commented 3 years ago

Hi, Thank you for your great work.

I tried to use NeuralRecon to reconstruct "Totem pole" data at here.

What I did is mimicking ScanNet data set style and test as a data of ScanNet. Therefore, I resized color images and changed file extension from "png" to "jpg" and get camera trajectory data. When I test using open3d to find out trajectory is correct, it seems nice.

However, It didn't work. and I find that the array occ has all negative values.

I want to know that what i did wrong and how to test data like "Totem pole".

Thanks.

JiamingSuen commented 3 years ago

We haven't experimented with this dataset before, but it should work. There are two general rules of thumb to adapt NeuralRecon to new datasets:

  1. Make sure the coordinate system and format of the camera poses are aligned with the ScanNet/ARKit demo.
  2. Make sure that the ground plane is correctly configured, as detailed in here.
DavidRecasens commented 2 years ago

Hi!

First congrats on your amazing work! I reopen this issue because I'm struggling with exactly the same problem: occ has all negative values when I ran it with my custom data.

I'm pretty sure the coordinate system and format of the camera poses are aligned with the ARKit demo. I think the problem is related with the offset that you apply to the x-y plane of the camera poses. I tried using an offset to put x-y plane at 1.5 meters over the ground, but it didn't work. With some random with some specific numbers NeuralRecon is able to reconstruct a couple of frames, but then the same problem appear.

I've seen in other issues that we have to configure the offset to simulate the conditions of ScanNet data during training, but you never specify which is that criterion. Therefore I wanted to ask you that, which is the criterion you follow to determine you need that +1.5 meters?

Thanks! :)