meshroomCL (colmap) fails on most image sets

[kjkroeker]

I'm trying MeshroomCL using an Intel Core I5 NUC (Gen6) with 16GB memory and the image sets on SSD.

Initially I tried the Monstree image set recommended elsewhere as a test for meshroom installations. I was able to process the 6 image set, but the full image set fails.

The failure occurs in MultiviewStereoCL (meshroomCL_colmap.exe ) and I get the "problem caused the program to stop working correctly" message.

Trying a small image set shot on my phone, I get the failure in Colmap but in this case I do get some failure information in the log. Referencing the log below: What do "Configuration has 2 problems..." and "!elems.empty() mean.

============================================================================== Image undistortion

Undistorting image [1/2] Undistorting image [2/2] Writing reconstruction... Writing configuration... Writing scripts... Elapsed time: 0.336 [minutes] Reading workspace... Reading configuration... Configuration has 2 problems... Elapsed time: 0.044 [minutes] Reading workspace... Reading configuration... Configuration has 2 problems... Elapsed time: 0.057 [minutes]

StereoFusion::Options

max_image_size: 3200 min_num_pixels: 5 max_num_pixels: 10000 max_traversal_depth: 100 max_reproj_error: 2 max_depth_error: 0.01 max_normal_error: 10 check_num_images: 50 cache_size: 32

Reading workspace... Reading configuration... Fusing image [1/2] in 16.881s (1247 points) Fusing image [2/2] in 1.547s (1247 points) Number of fused points: 1247 Elapsed time: 0.350 [minutes] Writing output: J:/Photogrammetry/ashby teapot/MeshroomCache/MultiviewStereoCL/4b4cc2bbf050cc58d6f42de26e2b41d03218cabb\fused.ply Triangulating points... Triangulation has 1236 using 1247 points. Initializing ray tracer... Initializing graph optimization... Integrating image [1/2] in 0.308s Integrating image [2/2] in 0.113s Setting up optimization... Running graph-cut optimization... Extracting surface as min-cut... Creating surface mesh model... F0917 11:24:34.404386 18760 math.h:233] Check failed: !elems.empty() Check failure stack trace: @ 00007FFE15455FCB public: void cdecl google::LogMessage::Flush(void) ptr64 @ 00007FFE15454B29 public: cdecl google::LogMessageFatal::~LogMessageFatal(void) ptr64 @ 00007FF6E533D401 (unknown) @ 00007FF6E535CB2A (unknown) @ 00007FF6E535CFBC (unknown) @ 00007FF6E521F849 (unknown) @ 00007FF6E52117A6 (unknown) @ 00007FF6E52541BF (unknown) @ 00007FF6E576EB24 (unknown) @ 00007FFEA9607034 BaseThreadInitThunk @ 00007FFEAB5E2651 RtlUserThreadStart

[revisionarian]

Hi @kjkroeker, thanks for posting your experience with MeshroomCL. I think that you have encountered two separate problems here:

1. MeshroomCL crashes with the full Monstree dataset (but works with the reduced version). My hunch is that you are running out of memory while processing the full dataset. Can you monitor your system's memory usage while MeshroomCL is running? Also, you can reduce the amount of memory required by reducing the "Max Image Size" and "Max Source Images" sliders in the MultiviewStereoCL node, although this may decrease the quality of the output.

I would be interested in seeing the log file when MeshroomCL crashes on this dataset. You should be able to find this file in the folder ...\MeshroomCache\MultiviewStereoCL\*\log where * corresponds to the sub-folder that was created with the same date and time that you ran that dataset. If you could post that log file here, we could confirm if lack of RAM is the problem.

2. The two images from your phone fail to reconstruct. The current version of MeshroomCL does not work with only two input images. For robustness, it requires the 3D points it creates to be seen from at least 3 cameras. Since none of the 3D points in your reconstructed mesh are seen from 3 views, the mesh is thus empty, resulting in the !elems.empty() error message. Obviously, we should implement a better way to alert the user of this constraint. If reconstructing 3D points from only two images is important to you, we could add an option to allow this.

The "Configuration has 2 problems..." message is just a way of saying that this dataset has 2 input images that need to be processed ("problem" = "task" in this context).

[kjkroeker]

I will look into the two suggestions you made, but I did monitor the memory use during execution and the application never went past 8GB utilization in the 16GB system. Also, the small image set I captured on my phone had 21 images, not 2 - so the 2 problems may have referred to 2 specific images. I will look over the images and see if any are iffy.

As to the log file, I will re-run the image set and post the log. I attempted the image set several times and I want a log file I know to be relevant.

[kjkroeker]

Here's the log file:

============================================================================== Image undistortion

Undistorting image [1/41] Undistorting image [2/41] Undistorting image [3/41] Undistorting image [4/41] Undistorting image [5/41] Undistorting image [6/41] Undistorting image [7/41] Undistorting image [8/41] Undistorting image [9/41] Undistorting image [10/41] Undistorting image [11/41] Undistorting image [12/41] Undistorting image [13/41] Undistorting image [14/41] Undistorting image [15/41] Undistorting image [16/41] Undistorting image [17/41] Undistorting image [18/41] Undistorting image [19/41] Undistorting image [20/41] Undistorting image [21/41] Undistorting image [22/41] Undistorting image [23/41] Undistorting image [24/41] Undistorting image [25/41] Undistorting image [26/41] Undistorting image [27/41] Undistorting image [28/41] Undistorting image [29/41] Undistorting image [30/41] Undistorting image [31/41] Undistorting image [32/41] Undistorting image [33/41] Undistorting image [34/41] Undistorting image [35/41] Undistorting image [36/41] Undistorting image [37/41] Undistorting image [38/41] Undistorting image [39/41] Undistorting image [40/41] Undistorting image [41/41] Writing reconstruction... Writing configuration... Writing scripts... Elapsed time: 1.242 [minutes] Reading workspace... Reading configuration... Configuration has 41 problems...

============================================================================================================================================================

==============================================================================Processing view 4 / 41

Processing view 3 / 41

Processing view 2 / 41

============================================================================== ==============================================================================Processing view 1 / 41

==============================================================================

==============================================================================

Reading inputs...

============================================================================== Processing view 5 / 41

PatchMatch::Problem

Reading inputs...ref_image_idx: 1 src_image_idxs: 5 28 2 11 23 19 37 3 35 30

PatchMatchOptions

max_image_size: 3200 gpu_index: 1 depth_min: 3.45505 depth_max: 17.4545 window_radius: 5 window_step: 1 sigma_spatial: 5 sigma_color: 0.2 num_samples: 15 ncc_sigma: 0.6 min_triangulation_angle: 1 incident_angle_sigma: 0.9 num_iterations: 5 geom_consistency: 0 geom_consistency_regularizer: 0.3 geom_consistency_max_cost: 3 filter: 1 filter_min_ncc: 0.1 filter_min_triangulation_angle: 3 filter_min_num_consistent: 2 filter_geom_consistency_max_cost: 1 write_consistency_graph: 0

PatchMatch::Run

OpenCL platform : Intel(R) OpenCL HD Graphics OpenCL device : Intel(R) Iris(R) Graphics 540

Reading inputs...PatchMatch::Problem

---

ref_image_idx: 3 src_image_idxs: 37 23 40 17 15 11 22 35 5 19

PatchMatchOptions

max_image_size: 3200 gpu_index: 3 depth_min: 4.02302 depth_max: 28.0851 window_radius: 5 window_step: 1 sigma_spatial: 5 sigma_color: 0.2 num_samples: 15 ncc_sigma: 0.6 min_triangulation_angle: 1 incident_angle_sigma: 0.9 num_iterations: 5 geom_consistency: 0 geom_consistency_regularizer: 0.3 geom_consistency_max_cost: 3 filter: 1 filter_min_ncc: 0.1 filter_min_triangulation_angle: 3 filter_min_num_consistent: 2 filter_geom_consistency_max_cost: 1 write_consistency_graph: 0

PatchMatch::Run

OpenCL platform : Intel(R) OpenCL HD Graphics OpenCL device : Intel(R) Iris(R) Graphics 540 Initialization: 2.90483s

Reading inputs... PatchMatch::Problem

ref_image_idx: 2 src_image_idxs: 28 33 1 26 24 19 5 14 9 32

PatchMatchOptions

max_image_size: 3200 gpu_index: 2 depth_min: 3.18788 depth_max: 9.7207 window_radius: 5 window_step: 1 sigma_spatial: 5 sigma_color: 0.2 num_samples: 15 ncc_sigma: 0.6 min_triangulation_angle: 1 incident_angle_sigma: 0.9 num_iterations: 5 geom_consistency: 0 geom_consistency_regularizer: 0.3 geom_consistency_max_cost: 3 filter: 1 filter_min_ncc: 0.1 filter_min_triangulation_angle: 3 filter_min_num_consistent: 2 filter_geom_consistency_max_cost: 1 write_consistency_graph: 0

PatchMatch::Run

OpenCL platform : Intel(R) OpenCL HD Graphics OpenCL device : Intel(R) Iris(R) Graphics 540

Reading inputs... PatchMatch::Problem

ref_image_idx: 0 src_image_idxs: 32 17 35 26 14 6 19 29 24 11

PatchMatchOptions

max_image_size: 3200 gpu_index: 0 depth_min: 3.27946 depth_max: 20.2708 window_radius: 5 window_step: 1 sigma_spatial: 5 sigma_color: 0.2 num_samples: 15 ncc_sigma: 0.6 min_triangulation_angle: 1 incident_angle_sigma: 0.9 num_iterations: 5 geom_consistency: 0 geom_consistency_regularizer: 0.3 geom_consistency_max_cost: 3 filter: 1 filter_min_ncc: 0.1 filter_min_triangulation_angle: 3 filter_min_num_consistent: 2 filter_geom_consistency_max_cost: 1 write_consistency_graph: 0

PatchMatch::Run

OpenCL platform : Intel(R) OpenCL HD Graphics OpenCL device : Intel(R) Iris(R) Graphics 540 Initialization: 5.9269s

PatchMatch::Problem

ref_image_idx: 4 src_image_idxs: 31 25 34 27 38 21 24 10 12 6

PatchMatchOptions

max_image_size: 3200 gpu_index: 4 depth_min: 3.60318 depth_max: 49.7427 window_radius: 5 window_step: 1 sigma_spatial: 5 sigma_color: 0.2 num_samples: 15 ncc_sigma: 0.6 min_triangulation_angle: 1 incident_angle_sigma: 0.9 num_iterations: 5 geom_consistency: 0 geom_consistency_regularizer: 0.3 geom_consistency_max_cost: 3 filter: 1 filter_min_ncc: 0.1 filter_min_triangulation_angle: 3 filter_min_num_consistent: 2 filter_geom_consistency_max_cost: 1 write_consistency_graph: 0

PatchMatch::Run

OpenCL platform : Intel(R) OpenCL HD Graphics OpenCL device : Intel(R) Iris(R) Graphics 540 Initialization: 9.98062s

[kjkroeker]

Got a new system and ran 0.7 against the full Monstree image set. It chewed through it with no problems.

Issue closed.