search

princeton-vl / DROID-SLAM

BSD 3-Clause "New" or "Revised" License
1.66k stars 273 forks source link

Problem with reconstruction result #63

Open lilly5791 opened 1 year ago

lilly5791 commented 1 year ago

Thank you for sharing such a great deep learning slam algorithm. I also thank you for the recently modified code which outputs the numpy pose results on reconstruction_path.

It works very well on my own data and the demo result is almost same with the groundtruth! However, I found a problem when I visualize the npy result with Matlab

As you can see in picture, sfm_bench and rgbd_dataset_freiburg3_cabinet trajectory visualization result is same with demo visualization. However my own data trajectory visualization result and the demo visualization is very different.

In my own data, the demo result is very good, but I don't understand why the matlab visualization result is bad even though the matlab code is same. Do you know why the npy result is different from the demo?

droid_slam_visualization

buenos-dan commented 1 year ago

Yes, I have the same problem with you. traj

Would you give me a contact method(WeChat or E-mail) therefore we can communicate more details.

lilly5791 commented 1 year ago

Here's my email 1913084@sookmyung.ac.kr

The movement is different from yours and mine but the droid slam result looks similar especially the round and noisy part.

buenos-dan commented 1 year ago

Try to save data directly from visualization.py and the result is good. Why reconstruction is bad? I guess poses were optimized after, so you should save the data in time and update data with dirty_index. PS: don't save tensor directly, use tensor.item() to save the value.

lilly5791 commented 1 year ago

Thank you for your advice. This worked well!

pranav-asthana commented 1 year ago

Hi can you please explain what you mean by save the data in time and update data with dirty_index?

lilly5791 commented 1 year ago

I think it's because the type of video.poses is torch.tensor and the pose information keeps changing. I'm not used to torch so I just saved all ix and pose as txt file and only used last lines for visualization.

senhuangpku commented 1 year ago

Hello, I have no idea about how to use the result of reconstruction_path, could you tell me your solution? Thanks!

pranav-asthana commented 1 year ago

This is what I am using. Just leaving this here in case it helps anyone.

  1. images, poses and depths are output for keyframes into reconstruction_path. These can be used with a reconstruction algorithm (like tsdf, a good implementation is in Open3D) or any other MVS system.
  2. In demo.py, traj_est stores the pose for each frame of the input video after global refinement and trajectory filling. If you need this information, you can save it to allow doing things like MVS on each input frame.
RitvikMandyam commented 1 year ago

This is what I am using. Just leaving this here in case it helps anyone.

  1. images, poses and depths are output for keyframes into reconstruction_path. These can be used with a reconstruction algorithm (like tsdf, a good implementation is in Open3D) or any other MVS system.
  2. In demo.py, traj_est stores the pose for each frame of the input video after global refinement and trajectory filling. If you need this information, you can save it to allow doing things like MVS on each input frame.

Could you share the code to do this, please? I'm working on a college project that uses DROID-SLAM, and I'd really like to show surfaces for my demo rather than a point cloud. I'm able to get pretty decent output by increasing filter_threshold in droid_slam/visualization.py, but it still looks terrible when you zoom into any part of the visualization.

surajiitd commented 1 year ago

@pranav-asthana Do you know why the reconstructions are saved before global-BA ? they should be saved after global-BA as the poses and the depths will be refined globally in global-BA https://github.com/princeton-vl/DROID-SLAM/blob/8016d2b9b72b101a3e9ac804ebf20b4c654dc291/demo.py#L131-L134