search

mapillary / OpenSfM

Open source Structure-from-Motion pipeline
https://www.opensfm.org/
BSD 2-Clause "Simplified" License
3.34k stars 852 forks source link

Reconstruction of a 360 degree view #114

Closed kevin-george closed 7 years ago

kevin-george commented 7 years ago

Hello,

I have been using this framework to reconstruct a 360 degree point of view(with enough parallax). My goal was to extract the estimated camera poses from this reconstruction. Questions ->

  1. Is there a way to prevent the outliers from being added to the final reconstruction? I see a few images that were incorrectly stitched and are labeled as NOT CONVERGED on the terminal.
  2. Is there a theoretical bound on the reconstruction that would prevent me from doing a 360 degree point of view? (E.g are the pictures being mapped to a plane instead of a sphere which would limit the reconstruction to the field of view of the camera?)

Any help would be appreciated, I haven't been able to find much in the way of documentation :-)

paulinus commented 7 years ago

@kevin-george the library supports 360 but it needs to know that the images are 360. I suspect that might be your problem. Citing from this other issue https://github.com/mapillary/OpenSfM/issues/77

Make sure images are detected as equirectangular. Look at the generated file camera_models.json and see if projection_type is equirectangular. If it is not, rename that file to camera_models_overrides.json and edit the projection type.

Please let me know if that worked for you. Thanks!

kevin-george commented 7 years ago

Thank you for your response Pau! I tried your suggestion and it improves the reconstruction, but it's not consistent. I see that running the bin/run_all on the same dataset with the same config.yaml gives different reconstructions at different times. Is this a known behavior?

sfuerte commented 7 years ago

I did see the inconsistency in the reconstruction with OpenCV (OpenSfM's dependency) 2.4 branch on a test workstation that had rather small amount of RAM (8GB if I remember correctly). After updating at least to the latest stable release - 3.1.0 - fixed the problem. Currently on their master branch - haven't noticed any issues.

On the second thought, because Debian packages are updated to newer versions quite/very slowly, I did complied Ceres from sources as well.

kevin-george commented 7 years ago

I am running OpenSfM with OpenCV 2.4.13.1(built from source) I didn't see any requirements for 3.1, is that the case? Or is that recommended?

sfuerte commented 7 years ago

I definitely didn't see it in the requirements; whether it's recommended or not, it's a question to @paulinus. I just shared my experience and what helped me to solve inconsistency issue.

PS: OpenCV was released to 3.2.0; their changelog is quite a lengthy one.

paulinus commented 7 years ago

@kevin-george OpenCV 3 is not required and we mainly use OpenCV 2.

OpenCV 3 should work just as OpenCV 2, but we are not testing it.

paulinus commented 7 years ago

@kevin-george could you share the dataset where you are seeing the random behavior?