search

cdcseacave / openMVS

open Multi-View Stereo reconstruction library
http://cdcseacave.github.io
GNU Affero General Public License v3.0
3.26k stars 898 forks source link

`TextureMesh` significantly slower when using non-zero `--resolution-level` #1094

Open xiaoxiae opened 7 months ago

xiaoxiae commented 7 months ago

Describe the bug The texturing of a mesh takes significantly more time when using a resolution level that is higher than zero (done to reduce memory consumption for larger scenes).

Resolution level 0:

15:40:20 [App     ] OpenMVS x64 v2.2.0
15:40:20 [App     ] Build date: Jan 25 2024, 10:11:25
15:40:20 [App     ] CPU: AMD Ryzen 7 1700 Eight-Core Processor           (16 cores)
15:40:20 [App     ] RAM: 62.74GB Physical Memory 32.00GB Virtual Memory
15:40:20 [App     ] OS: Linux 5.15.146-1-MANJARO (x86_64)
15:40:20 [App     ] Disk: 30.88GB (31.37GB) space
15:40:20 [App     ] SSE & AVX compatible CPU & OS detected
15:40:20 [App     ] Command line: TextureMesh -i /tmp/tmp9q3e4x60/06-LargeTexturedCheckpoint/project/mvs/scene_dense_mesh.mvs -w /tmp/tmp9q3e4x60/06-LargeTexturedCheckpoint/project/mvs -o /tmp/tmp9q3e4x60/06-LargeTexturedCheckpoint/project/mvs/tmp.mvs --mesh-file /tmp/tmp9q3e4x60/06-LargeTexturedCheckpoint/project/mvs/scene_dense_mesh.ply --resolution-level 0 --cuda-device -1
15:40:20 [App     ] MapSMtoCores for SM 8.6 is undefined; default to use 64 cores/SM
15:40:20 [App     ] CUDA device 0 initialized: NVIDIA GeForce RTX 3060 Ti (compute capability 8.6; memory 7.78GB)
15:40:21 [App     ] Scene loaded (149ms):
    5 images (5 calibrated) with a total of 26.58 MPixels (5.32 MPixels/image)
    0 points, 218075 vertices, 436005 faces
15:40:21 [App     ] Mesh loaded: 218075 vertices, 436005 faces (58ms)

Initialized views 1 (20.00%, 684ms, ETA 2s)...
Initialized views 2 (40.00%, 835ms, ETA 1s)...
Initialized views 3 (60.00%, 941ms, ETA 627ms)...
Initialized views 4 (80.00%, 1s, ETA 261ms)...   
Initialized views 5 (100.00%, 1s, ETA 0ms)... 
Initialized views 5 (100%, 1s174ms)          
15:40:33 [App     ] Assigning the best view to each face completed: 436005 faces (12s819ms)
15:40:38 [App     ] Generating texture atlas and image completed: 71 patches, 8192 image size (4s521ms)
15:40:38 [App     ] Mesh texturing completed: 218075 vertices, 436005 faces (17s347ms)
15:40:46 [App     ] Scene saved (7s989ms):
    5 images (5 calibrated)
    0 points, 218075 vertices, 436005 faces
15:40:54 [App     ] Mesh saved: 218075 vertices, 436005 faces (7s974ms)
15:40:54 [App     ] MEMORYINFO: {
15:40:54 [App     ]     VmPeak:  9692128 kB
15:40:54 [App     ]     VmSize:  8186904 kB
15:40:54 [App     ] } ENDINFO

Resolution level 3:

15:36:08 [App     ] OpenMVS x64 v2.2.0
15:36:08 [App     ] Build date: Jan 25 2024, 10:11:25
15:36:08 [App     ] CPU: AMD Ryzen 7 1700 Eight-Core Processor           (16 cores)
15:36:08 [App     ] RAM: 62.74GB Physical Memory 32.00GB Virtual Memory
15:36:08 [App     ] OS: Linux 5.15.146-1-MANJARO (x86_64)
15:36:08 [App     ] Disk: 31.03GB (31.37GB) space
15:36:08 [App     ] SSE & AVX compatible CPU & OS detected
15:36:08 [App     ] Command line: TextureMesh -i /tmp/tmpzo4a8ngj/06-LargeTexturedCheckpoint/project/mvs/scene_dense_mesh.mvs -w /tmp/tmpzo4a8ngj/06-LargeTexturedCheckpoint/project/mvs -o /tmp/tmpzo4a8ngj/06-LargeTexturedCheckpoint/project/mvs/tmp.mvs --mesh-file /tmp/tmpzo4a8ngj/06-LargeTexturedCheckpoint/project/mvs/scene_dense_mesh.ply --resolution-level 3 --cuda-device -1
15:36:08 [App     ] MapSMtoCores for SM 8.6 is undefined; default to use 64 cores/SM
15:36:08 [App     ] CUDA device 0 initialized: NVIDIA GeForce RTX 3060 Ti (compute capability 8.6; memory 7.78GB)
15:36:08 [App     ] Scene loaded (136ms):
    5 images (5 calibrated) with a total of 26.55 MPixels (5.31 MPixels/image)
    0 points, 196957 vertices, 393774 faces
15:36:08 [App     ] Mesh loaded: 196957 vertices, 393774 faces (52ms)

Initialized views 1 (20.00%, 321ms, ETA 1s)...
Initialized views 4 (80.00%, 434ms, ETA 108ms)...
Initialized views 5 (100%, 466ms)                
15:36:14 [App     ] Assigning the best view to each face completed: 393774 faces (6s343ms)
15:38:26 [App     ] Generating texture atlas and image completed: 89266 patches, 4096 image size (2m11s870ms)
15:38:26 [App     ] Mesh texturing completed: 196957 vertices, 393774 faces (2m18s302ms)
15:38:29 [App     ] Scene saved (2s299ms):
    5 images (5 calibrated)
    0 points, 196957 vertices, 393774 faces
15:38:30 [App     ] Mesh saved: 196957 vertices, 393774 faces (1s583ms)
15:38:30 [App     ] MEMORYINFO: {
15:38:30 [App     ]     VmPeak:  7779864 kB
15:38:30 [App     ]     VmSize:  7648792 kB
15:38:30 [App     ] } ENDINFO

This is very likely due to the fact that there are 71 patches in the resolution 0 and 89266 in resolution 3, which is strange since the code should arguably behave in the same way.

Is there something I'm missing?

xiaoxiae commented 7 months ago

Doing more experiments, I couldn't replicate this issue. Running the full reconstruction (sparse, dense, model, texture) for varying --resolution-levels decreased the time... I'll see if I can pin the issue down further.

cdcseacave commented 7 months ago

first of all you are not using the same input, you should make the experiemnt with changing only resolution level and nothiing else

second, all command line paths are relative to the working folder, no need to set full path for any argument, except the working folder; your command are very hard to parse and read

xiaoxiae commented 7 months ago

The input data is the same, that was the point of the issue. The reconstruction was, however, ran twice, so the vertices/faces are slightly different.

xiaoxiae commented 7 months ago

I'll close the issue for now, will reopen when I have some more concrete examples.

cdcseacave commented 7 months ago

what do u mean by the same if the first is

Mesh loaded: 218075 vertices, 436005 faces

and the second is

Mesh loaded: 196957 vertices, 393774 faces
xiaoxiae commented 7 months ago

The input data is the same (exactly same images). The reconstruction was then ran and so the meshes produced differ slightly. It was therefore strange that the differences in texturing were as large as I mentioned.

xiaoxiae commented 6 months ago

@cdcseacave just ran into the issue again, this time with the same input data (attached below).

It seems that --resolution-level 0 and --resolution-level 1 generate entirely different texture atlases, with the one with 0 being good (only 186 patches) whereas the one with 1 being really bad (~28000 patches; producing a large amount of untextured triangles).

This is very unintuitive since the packing theoretically shouldn't rely on this parameter (at least from my understanding).

input.zip output.zip (produced with TextureMesh -i scene.mvs --mesh-file scene_dense_mesh.ply --resolution-level 0 and --resolution-level 1 respectively)

res1 res0

0 1