InterfaceCOLMAP example

[ivanfarevalo]

Specifications like the version of the project, operating system, and hardware

16:43:15 [App ] Build date: Aug 11 2018, 12:41:00 16:43:15 [App ] CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz 16:43:15 [App ] RAM: 16.00GB Physical Memory 16.00GB Virtual Memory 16:43:15 [App ] OS: Darwin 17.4.0 (x86_64) 16:43:15 [App ] SSE & AVX compatible CPU & OS detected 16:43:15 [App ] Command line: rock.mvs

Steps to reproduce the problem

Hello! Thanks for keeping up with responding to issues, it has helped me troubleshoot a lot. I am using COLMAP to render the sparse cloud of a rock, and it looks great, it was able to position all 80 pictures accurately and get a good sparse cloud. I can successfully import the .nvm file with InterfaceVisualSFM, but problem is once try to render the dense point cloud, it tells me that non of the cameras have good images in view. I have tried with openMVG for the sparse cloud and gives me the same output.. I have also run the sample mvs file to make sure openMVS is working.

Thanks again, and id appreciate any feedback!

Here is a picture of my sparse cloud in COLMAP: https://ibb.co/iDMEAz https://ibb.co/nhSXxe https://ibb.co/hyjCxe ... 16:39:56 [App ] 80 cameras; 5429 3D points; 42042 projections Processed images 80 (100%, 2m28s832ms)
16:42:25 [App ] Scene saved to interface format (2ms): 80 images (80 calibrated) 5429 points, 0 vertices, 0 faces 16:42:25 [App ] Input data imported: 80 cameras, 80 poses, 80 images, 5429 vertices (2m28s904ms) ... 16:43:15 [App ] Image loaded 78: 2018-08-20_150721.jpg 16:43:15 [App ] Image loaded 79: 2018-08-20_151243.jpg 16:43:15 [App ] Scene loaded from interface format (22ms): 80 images (80 calibrated) with a total of 616.60 MPixels (7.71 MPixels/image) 5429 points, 0 vertices, 0 faces 16:43:22 [App ] Preparing images for dense reconstruction completed: 80 images (6s75ms) 16:43:22 [App ] error: reference image 0 has no good images in view 16:43:22 [App ] error: reference image 1 has no good images in view 16:43:22 [App ] error: reference image 2 has no good images in view 16:43:22 [App ] error: reference image 3 has no good images in view .. 16:43:22 [App ] error: reference image 77 has no good images in view 16:43:22 [App ] error: reference image 78 has no good images in view 16:43:22 [App ] error: reference image 79 has no good images in view 16:43:22 [App ] Selecting images for dense reconstruction completed: 0 images (133ms) Fused depth-maps 0 (100%, 0ms) 16:43:22 [App ] Depth-maps fused and filtered: 0 depth-maps, 0 depths, 0 points (-2147483648%%) (0ms) 16:43:22 [App ] Densifying point-cloud completed: 0 points (6s212ms) 16:43:22 [App ] Scene saved (7ms): 80 images (80 calibrated) 0 points, 0 vertices, 0 faces

[cdcseacave]

Not sure what might be wrong with the NVM project, but there is a importer for COLMAP, so why exporting to VisualSFM and after to OpenMVS?

[ivanfarevalo]

Thank you for the quick reply!

I have looked through other posts and never came upon a way to import the sparse cloud from COLMAP into openMVS without exporting it as .nvm file and then using InterfaceVisualSFM to get a .mvs file. Please let me know if there's a better way, also I was able to get 51 out of the 80 images to be used with the dense reconstruction.. I had used a small rock with a white background in every picture and had high shutter speed in my camera to avoid extracting unnecessary features. Sample Pictures: https://ibb.co/kxpUce https://ibb.co/fevPAz https://ibb.co/eRoNxe Today, I used a bigger subject in my photos as well as lowered the shutter speed of my camera, allowing it to extract features from where the rock is sitting and I was able to get 51 out of 80 images used for building the dense cloud. I might just use newspaper or other pattern under my rock and see if my results improve, as well as closer pictures.

[cdcseacave]

https://github.com/cdcseacave/openMVS/tree/develop/apps/InterfaceCOLMAP

[ivanfarevalo]

I appreciate the help!

I've been trying to use the COLMAP importer, but keep getting this error messages when reading from the image.txt file..

13:38:10 [App ] Build date: Aug 23 2018, 12:42:15 13:38:10 [App ] CPU: Intel(R) Core(TM) i5-7360U CPU @ 2.30GHz 13:38:10 [App ] RAM: 16.00GB Physical Memory 16.00GB Virtual Memory 13:38:10 [App ] OS: Darwin 17.4.0 (x86_64) 13:38:10 [App ] SSE & AVX compatible CPU & OS detected 13:38:10 [App ] Command line: InterfaceCOLMAP -i . -o ./model.mvs 13:38:10 [App ] Reading cameras: sparse/cameras.txt 13:38:10 [App ] Reading images: sparse/images.txt libc++abi.dylib: terminating with uncaught exception of type std::out_of_range: unordered_map::at: key not found Abort trap: 6

I have a total of 80 images, and i'm exporting the model as text into the sparse folder... Thanks again.

[cdcseacave]

InterfaceCOLMAP is meant to be used on the undistorted scene; basically the scene that is saved by COLMAP to start the dense reconstruction.

[ivanfarevalo]

So the process I am following in COLMAPS is creating new database, selecting picture folder, extracting and matching features, start sparse reconstruction, and then undistorting images. I can't do stereo reconstruction because I don't have CUDA. At this point I have a sparse folder with cameras.bin, images.bin, and points3d.bin. When I try to use InterfaceCOLMAP, it tries to look for sparse/cameras.txt. So I exported the model as text, and put cameras.txt, images.txt, and points3d.txt in the sparse folder. It is able to read cameras.txt but not images.txt, i get the error I mentioned above... Am I only able to use InterfaceCOLMAP on the scene created after running stereo reconstruction ? If not, what am I missing? my parameters are -i currentDir where sparse folder is stored, -o model.mvs, and --image_folder images/ . I am new to the topic so I apologize if it's something obvious ... thanks in advance

[cdcseacave]

InterfaceCOLMAP is meant to be used with the sparse folder inside dense folder. After COLMAP undistorts the images, it creates a dense folder and inside is the sparse folder and the images folder. Convert that in TXT, copy InterfaceCOLMAP inside dense folder and run: InterfaceCOLMAP . -o scene.mvs

[ivanfarevalo]

Thanks! that worked..

[CanCanZeng]

Hi @cdcseacave , I follwed your instruction, run InterfaceCOLMAP . -o scene.mvs under colmap's dense output folder , the file path structure is:

dense .....images (contains the undistorted images) .....sparse ..........cameras.txt ..........images.txt ..........points3D.txt .....stereo (contains some empty folders) .....run-colmap-geometric.sh .....run-colmap-photometric.sh

But the InterfaceCOLMAP.ext ceased when reading images.txt file, without any error output. Below is its log file:

15:10:37 [App ] Build date: Dec 21 2018, 18:08:41 15:10:37 [App ] CPU: Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz 15:10:37 [App ] RAM: 15.90GB Physical Memory 128.00TB Virtual Memory 15:10:37 [App ] OS: Windows 8 x64 15:10:37 [App ] SSE & AVX compatible CPU & OS detected 15:10:37 [App ] Command line: InterfaceCOLMAP . -o scene.mvs 15:10:37 [App ] Reading cameras: sparse/cameras.txt 15:10:37 [App ] Reading images: sparse/images.txt

I use the binary file on windows 10, both for colmap and openMVS.

[cdcseacave]

I coded this some time ago, maybe COLMAP changed something in the interface, or maybe I do not support all colmap features. Pls make sure the sparse scene you try to convert is the scene exported in TXT after undistortion (such the cameras are simple pinholes model). If that still does not work, pls debug it and see why it exits.

[CanCanZeng]

Hi, I tried the same operation on Ubuntu16.04, this time the colmap and openMVS are all compiled from source. The InterfaceCOLMAP module works! And I found that the problem is probably camera model, when I use colmap default model SIMPLE RADINAL , there is an error

23:35:21 [App     ] Build date: Jan 30 2019, 17:14:12
23:35:21 [App     ] CPU: Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz
23:35:21 [App     ] RAM: 7.70GB Physical Memory 9.31GB Virtual Memory
23:35:21 [App     ] OS: Linux 4.4.90-040490-lowlatency (x86_64)
23:35:21 [App     ] SSE & AVX compatible CPU & OS detected
23:35:21 [App     ] Command line: InterfaceCOLMAP -i /home/zcc/Dataset/SfM/ImageDataset_SceauxCastle/resultOfColmap3/dense --image-folder /home/zcc/Dataset/SfM/ImageDataset_SceauxCastle/resultOfColmap3/dense/images
23:35:21 [App     ] Reading cameras: /home/zcc/Dataset/SfM/ImageDataset_SceauxCastle/resultOfColmap3/dense/sparse/cameras.txt
23:35:21 [App     ] Reading images: /home/zcc/Dataset/SfM/ImageDataset_SceauxCastle/resultOfColmap3/dense/sparse/images.txt
terminate called after throwing an instance of 'std::out_of_range'
  what():  _Map_base::at
Aborted

but when I change the model to PINHOLE, it works.

Can you support the default SIMPLE RADINAL model? The PINHOLE model sometimes too simple since it ignores distortion.

I also find another bug, when I use the parameter --archive-type 0 , it's not working, it always output an binary scene.mvs file.

[cdcseacave]

You can use COLMAP with any radial distortion you want, but at the end you need to undistort the images and save the scene in TXT format. Please see COLMAP documentation on how to do that

[CanCanZeng]

Yes, I have undistorted the images. I find that if I change the cameras.txt it will work.

# Camera list with one line of data per camera:
#   CAMERA_ID, MODEL, WIDTH, HEIGHT, PARAMS[]
# Number of cameras: 1
1 SIMPLE_RADIAL 2832 2128 2969.56 1416 1064 -0.164225
#1 PINHOLE 2832 2128 2969.56 2969.56 1416 1064

above is the content of cameras.txt file. If I comment the SIMPLE_RADIAL line and uncomment the PINHOLE line, it runs smoothly.

By the way, is the InterfaceCOLMAP only support sparse reconstruct of colmap? Is there any way to import the dense point cloud of colmap? I see lots of people are trying to do this, since we find that the dense cloud of colmap some times more clean but the mesh module of openMVS is rather better. Thank you for your reply anyway @cdcseacave !

[cdcseacave]

after COLMAP undistorts the images, it saves a new scene with PINHOLE cameras, you have to use that one instead

InterfaceCOLMAP is already able to import dense point cloud from COLMAP, exactly the same way as you do it for sparse, but once the dense one computed and saved

[CanCanZeng]

Yes, thank you for your help @cdcseacave , InterfaceCOLMAP can import the dense result of COLMAP. But I find that the dense result of COLMAP does not contain any information about original images, that is to say, the images.txt of dense result is empty, I don't know why COLMAP have this strange behavior.

What the InterfaceCOLMAP needs? I'm trying to export COLMAP's dense result myself.

[CanCanZeng]

Good news, I find how to import COLMAP's dense result! Just modify the camera.txt and images.txt exported by COLMAP. I copy the cameras.txt and images.txt of sparse result and modify them like this:

cameras.txt:

# Camera list with one line of data per camera:
#   CAMERA_ID, MODEL, WIDTH, HEIGHT, PARAMS[]
# Number of cameras: 1
1 PINHOLE 2832 2128 2969.56 2969.56 1416 1064

images.txt

# Image list with two lines of data per image:
#   IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME
#   POINTS2D[] as (X, Y, POINT3D_ID)
# Number of images: 0, mean observations per image: 0
9 0.964011 -0.0168924 0.26125 -0.0463356 -3.86388 -0.0940421 0.0708848 1 100_7108.JPG

2 0.993355 -0.000633085 -0.114596 0.0106212 4.43491 0.271643 1.8434 1 100_7101.JPG

1 0.983958 -0.0107727 -0.175106 0.0323667 6.29032 0.332248 1.81498 1 100_7100.JPG

11 0.93218 0.0430797 0.352149 -0.0719385 -6.36992 0.0890445 -0.820477 1 100_7110.JPG

10 0.941212 -0.0183183 0.333339 -0.0516724 -5.16774 -0.236679 0.0687572 1 100_7109.JPG

3 0.998112 0.0189198 -0.0584155 0.00132721 3.18407 0.26028 1.85184 1 100_7102.JPG

4 0.999856 0.00123509 -0.0169414 0.00011572 2.45281 0.331175 1.61664 1 100_7103.JPG

5 0.998625 0.00992314 0.0510467 -0.00661545 1.18847 0.310698 1.55393 1 100_7104.JPG

6 0.995465 0.00180184 0.0940824 -0.0139677 -0.0606435 0.305068 1.48994 1 100_7105.JPG

7 0.98966 0.00137992 0.142286 -0.0180354 -1.21575 0.212448 1.21263 1 100_7106.JPG

8 0.974046 -0.0272825 0.222154 -0.0337248 -2.40192 0.0728573 0.615798 1 100_7107.JPG

Hopefully this will be helpfull for any one who wants to conbine the dense reconstuction of COLMAP with openMVS.

Below is the comparison of meshes produced by openMVS and COLMAP, both with default parameters.

[Chawin-S]

For those who have the same issue as mine and in this topic to use dense point cloud of Colmap and then do the meshing in OpenMVS, To sum up for you, the function "InterfaceCOLMAP" from OpenMVS is able to import the dense point cloud of Colmap and convert it to .mvs format Method and command are the same as when you import and convert the sparse point cloud but after finish densifying.

The main difference is that after you finish the densify in Colmap, fuse.ply and fuse.ply.vis (output of densifying) will be created. The function "InterfaceCOLMAP" will automatically check and do its work. if there is no fuse.ply and fuse.ply.vis , it will import and convert out the sparse points However, if there are fuse.ply and fuse.ply.vis , it will import and convert out the dense points

Thank you all contributors of Colmap and OpenMVS

I have quote the reply of cdcseacave here

after COLMAP undistorts the images, it saves a new scene with PINHOLE cameras, you have to use that one instead

InterfaceCOLMAP is already able to import dense point cloud from COLMAP, exactly the same way as you do it for sparse, but once the dense one computed and saved

Relate issues https://github.com/cdcseacave/openMVS/issues/360 https://github.com/colmap/colmap/issues/557 https://github.com/colmap/colmap/issues/157

[cdcseacave]

Happy to see you get good results with COLMAP+OpenMVS. However as far as I know the latest version of OpenMVS dense point-cloud is much better compared to the COLMAP version; pls see the leaderboard on the two most representative benchmarks in this field:

• ETH3D Benchmark (created by the COLMAP authors)
• Tanks and Temples Benchmark (created by Intel)

Can you pls point me to the cases where COLMAP still performs better than OpenMVS? (the source images would be helpful for me to correct the problem)

[Chawin-S]

Thank you for your information. I am just curious because sometimes I see that the dense point cloud from OpenMVS is noisier around the object with the default setting, however, most of them and also the far away background as well will be discarded in the meshing process . I don't know much about which parameters to adjust. I believe that with proper parameters, the mesh result will be better as you said. Here are the screenshot of

1. Dense points from OpenMVS and its mesh

2. Dense points from Colmap and it's mesh

and Here are the source images in case you have some time to look at it inputimage.zip

Thank you

[cdcseacave]

Wow, indeed, OpenMVS does not seem to perform well on these type of images. Last time I saw digital cameras taking RGB photos of so low resolution (~500pixels width) and on top of that so low quality (actually I do not remember seeing noisier photos) was maybe in my childhood, ~20 years ago. Actually I was not even able to reproduce the results you display above, with the latest OpenMVS I get much worse results. COLMAP does a great job in this case (amazing I would say). Due to lack of time and low frequency of this type of input, I do not plan to support these type of images. Thank you for reporting this.

[Chawin-S]

I agree with you. Only for low resolution images that OpenMVS shows worse. With the high resolution images, OpenMVS creates better dense points with less processing time than Colmap. Too bad I have to work with light field image. It has a trade off between spatial resolution and angular resolution. That's why my input is about ~500 x ~300 px

[cdcseacave]

Light field camera, that explains it. For light field cameras there are specialized algorithms that not only should perform better but also faster. Please search on the net, I know several papers on this topic, but no opensource software.

[elliestath]

Hi, I am stil having the std:: out_of_range problem while trying to use the InterfaceCOLMAP. I am sure my folder structure is correct, since for some datasets it works, but for most of them I get:

14:24:55 [App ] Command line: InterfaceCOLMAP -i . -o sparse.mvs 14:24:55 [App ] Reading cameras: sparse/cameras.txt 14:24:55 [App ] Reading images: sparse/images.txt terminate called after throwing an instance of 'std::out_of_range' what(): _Map_base::at Aborted (core dumped)

I use the undistorted images I get from COLMAP, and the .txt files under the dense reconstruction folder. I use scaled images (i.e to half resolution) but I am using it from the beginning of the pipeline in COLMAP, so I guess this shouldn't be a problem. I tried also to use --max_image _size while using the image_undistorter of COLMAP, it makes no difference.

[cdcseacave]

great, but how can I help you with no debug data?

[elliestath]

points3D.txt cameras.txt

sure, I am using the Tanks_and_Temples dataset and more specifically the Barn

SfM is done in COLMAP. In this dataset I do not scale down the images. Unfortunately I cannot upload my undistorted images neither my images.txt file since they are very large.

[cdcseacave]

I do not need images of any kind, just the 3 txt files

[elliestath]

cameras https://ufile.io/wzs7ux84 images https://ufile.io/y0fdzqmt points3D https://ufile.io/pv4cfy27

Let me know if you have any problems

[cdcseacave]

the data you sent above work for me:

20:56:51 [App     ] Build date: Oct 28 2019, 13:44:08
20:56:51 [App     ] CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz (12 cores)
20:56:51 [App     ] RAM: 31.89GB Physical Memory 128.00TB Virtual Memory
20:56:51 [App     ] OS: Windows 8 x64
20:56:51 [App     ] SSE & AVX compatible CPU & OS detected
20:56:51 [App     ] Command line: InterfaceCOLMAP .
20:56:51 [App     ] Reading cameras: sparse/cameras.txt
20:56:51 [App     ] Reading images: sparse/images.txt
20:56:53 [App     ] Reading points: sparse/points3D.txt
20:56:53 [App     ] Exported data: 410 images & 86181 vertices (2s842ms)
20:56:53 [App     ] MEMORYINFO: {
20:56:53 [App     ]     PageFaultCount 12682
20:56:53 [App     ]     PeakWorkingSetSize 36.63MB
20:56:53 [App     ]     WorkingSetSize 8.91MB
20:56:53 [App     ]     QuotaPeakPagedPoolUsage 238.41KB
20:56:53 [App     ]     QuotaPagedPoolUsage 238.24KB
20:56:53 [App     ]     QuotaPeakNonPagedPoolUsage 12.48KB
20:56:53 [App     ]     QuotaNonPagedPoolUsage 12.21KB
20:56:53 [App     ]     PagefileUsage 3.22MB
20:56:53 [App     ]     PeakPagefileUsage 32.36MB
20:56:53 [App     ] } ENDINFO

[elliestath]

That's odd as I am continuing having the same result. I am running it on LInux.

`10:20:26 [App     ] Build date: Jun  5 2019, 12:46:44
10:20:26 [App     ] CPU: AMD Ryzen Threadripper 2950X 16-Core Processor  (32 cores)
10:20:26 [App     ] RAM: 31.35GB Physical Memory 2.00GB Virtual Memory
10:20:27 [App     ] OS: Linux 4.15.0-66-generic (x86_64)
10:20:27 [App     ] SSE & AVX compatible CPU & OS detected
10:20:27 [App     ] Command line: InterfaceCOLMAP .
10:20:27 [App     ] Reading cameras: sparse/cameras.txt
10:20:27 [App     ] Reading images: sparse/images.txt
terminate called after throwing an instance of 'std::out_of_range'
  what():  _Map_base::at
Aborted (core dumped)`

[elliestath]

after pulling the latest version of the file InterfaceCOLMAP.cpp problem is solved! Thanks for the support!

[Truthonlyone]

Good news, I find how to import COLMAP's dense result! Just modify the camera.txt and images.txt exported by COLMAP. I copy the cameras.txt and images.txt of sparse result and modify them like this:

cameras.txt:

# Camera list with one line of data per camera:
#   CAMERA_ID, MODEL, WIDTH, HEIGHT, PARAMS[]
# Number of cameras: 1
1 PINHOLE 2832 2128 2969.56 2969.56 1416 1064

images.txt

# Image list with two lines of data per image:
#   IMAGE_ID, QW, QX, QY, QZ, TX, TY, TZ, CAMERA_ID, NAME
#   POINTS2D[] as (X, Y, POINT3D_ID)
# Number of images: 0, mean observations per image: 0
9 0.964011 -0.0168924 0.26125 -0.0463356 -3.86388 -0.0940421 0.0708848 1 100_7108.JPG

2 0.993355 -0.000633085 -0.114596 0.0106212 4.43491 0.271643 1.8434 1 100_7101.JPG

1 0.983958 -0.0107727 -0.175106 0.0323667 6.29032 0.332248 1.81498 1 100_7100.JPG

11 0.93218 0.0430797 0.352149 -0.0719385 -6.36992 0.0890445 -0.820477 1 100_7110.JPG

10 0.941212 -0.0183183 0.333339 -0.0516724 -5.16774 -0.236679 0.0687572 1 100_7109.JPG

3 0.998112 0.0189198 -0.0584155 0.00132721 3.18407 0.26028 1.85184 1 100_7102.JPG

4 0.999856 0.00123509 -0.0169414 0.00011572 2.45281 0.331175 1.61664 1 100_7103.JPG

5 0.998625 0.00992314 0.0510467 -0.00661545 1.18847 0.310698 1.55393 1 100_7104.JPG

6 0.995465 0.00180184 0.0940824 -0.0139677 -0.0606435 0.305068 1.48994 1 100_7105.JPG

7 0.98966 0.00137992 0.142286 -0.0180354 -1.21575 0.212448 1.21263 1 100_7106.JPG

8 0.974046 -0.0272825 0.222154 -0.0337248 -2.40192 0.0728573 0.615798 1 100_7107.JPG

Hopefully this will be helpfull for any one who wants to conbine the dense reconstuction of COLMAP with openMVS.

Below is the comparison of meshes produced by openMVS and COLMAP, both with default parameters.

Hey bro, Recently I came up with this problem too. And I found the the reason is that the 'Fusion' button is clicked. When I export text files after click the 'Stereo' button and before click 'Fusion' button, the content of cameras.txt and images.txt is exist, but when I export text files after click the 'Fusion' button, those two files was empty. So, just export the text files before click the 'Fusion' button~

[cdcseacave]

the files generated by COLMAP should not be edited by hand (or in any way); the correct way to use COLMAP data is to use the sparse scene exported for densification, that has the Pinhole camera model and undistorted images.