cdcseacave / openMVS

open Multi-View Stereo reconstruction library
http://cdcseacave.github.io
GNU Affero General Public License v3.0
3.18k stars 890 forks source link

Densify point cloud "reference image $number$ has no good images in view error #560

Open sangukbo opened 4 years ago

sangukbo commented 4 years ago

Hi,

I am trying to use DensifyPoinitCloud function, but I keep getting the following error.

(base) ubuntu_16_04@ubuntu_16_04:~/gitreconstruction/image_pylon/reconstruction_output/reconstruction_sequential$ ~/gitreconstruction/openMVS_build/bin/DensifyPointCloud scene.mvs
15:17:29 [App     ] Build date: Apr 29 2020, 11:50:26  
15:17:29 [App     ] CPU: Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz (8 cores)  
15:17:29 [App     ] RAM: 15.49GB Physical Memory 0B Virtual Memory  
15:17:29 [App     ] OS: Linux 4.13.0-041300-generic (x86_64)  
15:17:29 [App     ] SSE & AVX compatible CPU & OS detected  
15:17:29 [App     ] Command line: scene.mvs
15:17:29 [App     ] Camera model loaded: platform 0; camera  0; f 3.731x3.731; poses 2
15:17:29 [App     ] Image loaded   0: left.png
15:17:29 [App     ] Image loaded   1: right.png
15:17:29 [App     ] Scene loaded from interface format (20ms):
    2 images (2 calibrated) with a total of 6.00 MPixels (3.00 MPixels/image)
    4700 points, 0 vertices, 0 faces
15:17:29 [App     ] Preparing images for dense reconstruction completed: 2 images (146ms)
15:17:29 [App     ] error: reference image   0 has no good images in view
15:17:29 [App     ] error: reference image   1 has no good images in view
15:17:29 [App     ] Selecting images for dense reconstruction completed: 0 images (5ms)
Fused depth-maps 0 (100%, 0ms)
15:17:29 [App     ] Depth-maps fused and filtered: 0 depth-maps, 0 depths, 0 points (-2147483648%%) (0ms)
15:17:29 [App     ] Densifying point-cloud completed: 0 points (165ms)
15:17:29 [App     ] Scene saved (0ms):
    2 images (2 calibrated)
    0 points, 0 vertices, 0 faces
15:17:29 [App     ] MEMORYINFO: {
15:17:29 [App     ]     VmPeak:  1201732 kB
15:17:29 [App     ]     VmSize:  1136196 kB
15:17:29 [App     ] } ENDINFO

The task that I am trying to do is the following.

  1. I have done SfM using openMVG using 200 images. There were about 200000 landmarks generated in this process.

  2. Among 200 images, I am interested in doing MVS with only two of the images ("left.png" and "right.png").

  3. I modified sfm_data.json file from the openMVG such that it only contains two views that I am interested ("left.png" and "right.png"). I also included only two extrinsics for the views. I kept the "structure" and "control_points" in sfm_data.json the same (containing about 200000 landmarks as they were).

  4. I converted the new sfm_data.json file to .mvs file. The process captures 4700 landmarks that are relevant to the two views.

  5. Then I performed DensifyPointCloud. I got the error that says that image 0 and image 1 ("left.png" and "right.png") has no good images in view.

I am confused because I thought 4700 landmarks would be sufficient for performing DensifyPointCloud for two images. I performed DensifyPointCloud with about 1000 landmarks with two images before and it worked okay.

I used ./viewer to check if the camera direction was correct and it captured enough landmarks (or points). It seems like the camera view is okay. I have attached the view image as view.png.

view

I have included the two images and .mvs file just in case.

I am trying to use 2 images among 200 images (that I have used when I did sfm), because I only need small part of the 3-D model really accurate, and I do not need to do that for the whole set of 200 images. The accurate MVS computation for the small part of the 3-D model needs to be done fast enough.

I thought using 4700 landmarks would be enough to perform MVS with two images, because I have ran openmvs with 1000 landmarks using two images.

Could you please help me in finding out how to resolve this issue?

In addition, I was also wondering if there is a way to open and modify .mvs file. For instance, sfm_data.bin file created in openmvg can be converted into .json file and I could modify the file. I was wondering if this would be possible for .mvs file and if so, I would like to know how.

I have attached the view.png, left.png, right.png, scene.mvs, and log file in openmvs.tar.gz file.

openmvs.tar.gz

I am using the most recent version of openmvs on ubuntu 16.04 machine.

Thank you very much for your help.

cdcseacave commented 4 years ago

Usually this error is given when an image does not have any appropriate neighbors that allows for estimating a good depth-map (ex good baseline, good scale, angle, etc). In this particular case the problem seems to be very small baseline. Of course you can run the densification algo on these images as well, even if small baseline, but the result is in general very noisy due to high imprecision for far away points. Feel free to play with disabling the baseline check in the code (SelectViews()). And in general is good to use a small focal length, do not zoom but get closer to the scene.

On the other hand, the densification algorithm implemented in OpenMVS is for general images, while these seem to be stereo images. For stereo images I recently implemented a SGM algorithm inside OpenMVS, and you should give it a try.

sangukbo commented 4 years ago

Thank you for the quick reply. I now understand the code much better and it has been really useful. I just have small follow up questions.

About disabling SelectViews(), I modified SceneDesify.cpp file and built the openmvs again. I just wanted to check if this way is correct, or there is other smarter way that has been implemented already.

In addition, I would like to try sgm algorithm that has been implemented, but it seems like I can't find an example to use the sgm algorithm. I was wondering if you could share more on how to use sgm algorithm (even just brief outline). I just wanted to check if there is an example to start from.

Thank you very much for your help.

cdcseacave commented 4 years ago

did you try the command line described in the wiki and did not work?

CeciliaPYY commented 4 years ago

wiki

meet same problem as @sangukbo, and what do you mean try the command? If you do mean the following command, I tried by still did't work well,

DensifyPointCloud scene.mvs --fusion-mode -1 DensifyPointCloud scene.mvs --fusion-mode -2

And here is my DensifyPointCloud.log

  1. result of DensifyPointCloud scene.mvs 14:59:29 [App ] Build date: May 20 2020, 12:54:10 14:59:29 [App ] CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz (2 cores) 14:59:29 [App ] RAM: 7.79GB Physical Memory 1024.00MB Virtual Memory 14:59:29 [App ] OS: Linux 4.9.184-linuxkit (x86_64) 14:59:29 [App ] SSE & AVX compatible CPU & OS detected 14:59:29 [App ] Command line: /dataset/home/reconstruction/scene.mvs 14:59:29 [App ] Camera model loaded: platform 0; camera 0; f 0.001x0.001; poses 2 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1996.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1997.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1998.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1999.JPG' 14:59:29 [App ] Image loaded 4: IMG_2001.JPG 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2002.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2003.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2004.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2005.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2006.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2007.JPG' 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2008.JPG' 14:59:29 [App ] Image loaded 12: IMG_2009.JPG 14:59:29 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2010.JPG' 14:59:29 [App ] Scene loaded from interface format (19ms): 14 images (2 calibrated) with a total of 23.26 MPixels (11.63 MPixels/image) 845 points, 0 vertices, 0 faces 14:59:29 [App ] Preparing images for dense reconstruction completed: 14 images (119ms) 14:59:29 [App ] error: reference image 4 has no good images in view 14:59:29 [App ] error: reference image 12 has no good images in view 14:59:29 [App ] Selecting images for dense reconstruction completed: 0 images (2ms) 14:59:29 [App ] Depth-maps fused and filtered: 0 depth-maps, 0 depths, 0 points (-2147483648%%) (0ms) 14:59:29 [App ] Densifying point-cloud completed: 0 points (129ms) 14:59:29 [App ] Scene saved (9ms): 14 images (2 calibrated) 0 points, 0 vertices, 0 faces 14:59:29 [App ] MEMORYINFO: { 14:59:29 [App ] VmPeak: 853180 kB 14:59:29 [App ] VmSize: 787644 kB 14:59:29 [App ] } ENDINFO

  2. result of fusion mode 15:14:56 [App ] Build date: May 20 2020, 12:54:10 15:14:56 [App ] CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz (2 cores) 15:14:56 [App ] RAM: 7.79GB Physical Memory 1024.00MB Virtual Memory 15:14:56 [App ] OS: Linux 4.9.184-linuxkit (x86_64) 15:14:56 [App ] SSE & AVX compatible CPU & OS detected 15:14:56 [App ] Command line: /dataset/home/reconstruction/scene.mvs --fusion-mode -1 15:14:56 [App ] Camera model loaded: platform 0; camera 0; f 0.001x0.001; poses 2 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1996.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1997.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1998.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1999.JPG' 15:14:56 [App ] Image loaded 4: IMG_2001.JPG 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2002.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2003.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2004.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2005.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2006.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2007.JPG' 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2008.JPG' 15:14:56 [App ] Image loaded 12: IMG_2009.JPG 15:14:56 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2010.JPG' 15:14:56 [App ] Scene loaded from interface format (29ms): 14 images (2 calibrated) with a total of 23.26 MPixels (11.63 MPixels/image) 845 points, 0 vertices, 0 faces 15:14:56 [App ] Preparing images for dense reconstruction completed: 14 images (142ms) 15:14:56 [App ] error: reference image 12 has no good images in view 15:14:56 [App ] error: reference image 12 has no good images in view 15:14:56 [App ] Selecting images for dense reconstruction completed: 0 images (1ms) 15:14:56 [App ] Densifying point-cloud completed: 845 points (149ms) 15:14:56 [App ] Scene saved (4ms): 14 images (2 calibrated) 845 points, 0 vertices, 0 faces 15:14:56 [App ] Point-cloud saved: 845 points (2ms) 15:14:56 [App ] MEMORYINFO: { 15:14:56 [App ] VmPeak: 869572 kB 15:14:56 [App ] VmSize: 804036 kB 15:14:56 [App ] } ENDINFO

15:15:05 [App ] Build date: May 20 2020, 12:54:10 15:15:05 [App ] CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz (2 cores) 15:15:05 [App ] RAM: 7.79GB Physical Memory 1024.00MB Virtual Memory 15:15:05 [App ] OS: Linux 4.9.184-linuxkit (x86_64) 15:15:05 [App ] SSE & AVX compatible CPU & OS detected 15:15:05 [App ] Command line: /dataset/home/reconstruction/scene.mvs --fusion-mode -2 15:15:05 [App ] Camera model loaded: platform 0; camera 0; f 0.001x0.001; poses 2 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1996.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1997.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1998.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_1999.JPG' 15:15:05 [App ] Image loaded 4: IMG_2001.JPG 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2002.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2003.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2004.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2005.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2006.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2007.JPG' 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2008.JPG' 15:15:05 [App ] Image loaded 12: IMG_2009.JPG 15:15:05 [App ] warning: uncalibrated image '/dataset/home/reconstruction/openmvs_images/IMG_2010.JPG' 15:15:05 [App ] Scene loaded from interface format (22ms): 14 images (2 calibrated) with a total of 23.26 MPixels (11.63 MPixels/image) 845 points, 0 vertices, 0 faces 15:15:05 [App ] Preparing images for dense reconstruction completed: 14 images (125ms) 15:15:05 [App ] error: reference image 12 has no good images in view 15:15:05 [App ] error: reference image 4 has no good images in view 15:15:05 [App ] Selecting images for dense reconstruction completed: 0 images (1ms) 15:15:05 [App ] Depth-maps fused and filtered: 0 depth-maps, 0 depths, 0 points (-2147483648%%) (0ms) 15:15:05 [App ] Densifying point-cloud completed: 0 points (142ms) 15:15:05 [App ] Scene saved (2ms): 14 images (2 calibrated) 0 points, 0 vertices, 0 faces 15:15:05 [App ] MEMORYINFO: { 15:15:05 [App ] VmPeak: 869572 kB 15:15:05 [App ] VmSize: 804036 kB 15:15:05 [App ] } ENDINFO

cdcseacave commented 4 years ago

check in Viewer the MVS file exported from OpenMVG