search

TRI-ML / packnet-sfm

TRI-ML Monocular Depth Estimation Repository
https://tri-ml.github.io/packnet-sfm/
MIT License
1.23k stars 242 forks source link

generating 3D scene from image sequence #170

Closed iariav closed 2 years ago

iariav commented 3 years ago

Hi, thanks for the amazing work. I was wondering if you have a script to generate a 3D reconstruction from an image sequence as you demonstrate in your video here - https://www.youtube.com/watch?v=b62iDkLgGSI (from 0:55)

it would be very appreciated. thanks

ZeyingXuHuaWei commented 3 years ago

also want to know

morsingher commented 2 years ago

Hi @iariav and @ZeyingXuHuaWei,

I'm not involved in this specific project, but glad to help. Once you have the image and the depth map, you just need to back-project each pixel in 3D with the simple equation:

X = depth inv(intrinsics) pixel

This is for a single image. If you want to accumulate multiple depth maps along a video sequence, make sure all the 3D points refer to the same frame (e.g. the pose of the first image). You should have a look at this work: https://github.com/Brummi/MonoRec. They do a similar thing and provide code for 3D reconstruction as well.

Hope it helps!

iariav commented 2 years ago

Thanks @morsingher for your help, but I don't understand why the authors ignore this issue for so long.. would help a lot to see the script they used instead of trying to incorporate someone else's to work with this repo

VitorGuizilini-TRI commented 2 years ago

I apologize for the delay. I have just open-sourced our visualization tool, it's here. I hope that helps! We will add more functionalities over time to generate the videos and plots we use in our publications.