Segfault and valgrind errors

[nh2]

With the versions from https://github.com/nmoehrle/mvs-texturing/issues/120#issuecomment-541404178, I sometimes (nondeterministically, see also #124) get segfault, and valgrind reports Invalid read of size 4:

``` # OMP_THREAD_LIMIT=1 valgrind bin/texrecon scene/::undistorted ~/small2.ply ~/texrecon-out-small-new2/outprefix --keep_unseen_faces --outlier_removal=gauss_damping --skip_geometric_visibility_test ==20303== Memcheck, a memory error detector ==20303== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==20303== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==20303== Command: bin/texrecon scene/::undistorted /root/small2.ply /root/texrecon-out-small-new2/outprefix --keep_unseen_faces --outlier_removal=gauss_damping --skip_geometric_visibility_test ==20303== bin/texrecon (built on Jan 1 1970, 00:00:01) Load and prepare mesh: PLY Loader: comment VCGLIB generated Reading PLY: 5150 verts... 5937 faces... done. Generating texture views: Initializing scene with 2215 views... Initialized 2215 views (max ID is 2214), took 2071ms. Loading 100%... done. (Took 0.54s) Building adjacency graph: Adding edges 100%... done. (Took 0.232s) 7068 total edges. View selection: Building BVH from 5937 faces... done. (Took: 617 ms) Calculating face qualities 100%... done. (Took 219.853s) Postprocessing face infos 100%... done. (Took 27.302s) Maximum quality of a face within an image: 1136.17 Clamping qualities to 67.0405 within normalization. Writing data cost file... done. Optimizing: Time[s] Energy 0 2462 2 2432 3 2395 3 2369 4 2364 4 2362 5 2361 6 2360 7 2360 7 2360 8 2360 9 2360 9 2360 10 2360 1 faces have not been seen Took: 262.587s Generating texture patches: Running... done. (Took 69.704s) 366 texture patches. Running global seam leveling: Create matrices for optimization... done. Lhs dimensionality: 5191 x 5191 Calculating adjustments: Color channel 0: CG took 31 iterations. Residual is 8.52602e-05 Color channel 1: CG took 31 iterations. Residual is 8.46199e-05 Color channel 2: CG took 31 iterations. Residual is 8.38964e-05 Took 0.184 seconds Adjusting texture patches 100%... done. (Took 8.399s) Running local seam leveling: Blending texture patches 100%...==20303== Invalid read of size 4 ==20303== at 0x549136: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== Address 0x7d8abc8 is 4 bytes after a block of size 36 alloc'd ==20303== at 0x4C2D1AF: operator new(unsigned long) (in valgrind-3.13.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==20303== by 0x54AAA7: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== ==20303== Invalid read of size 8 ==20303== at 0x5487FF: simple_laplacian(int, std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x549217: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== Address 0x7f651f8 is 12 bytes after a block of size 108 alloc'd ==20303== at 0x4C2D1AF: operator new(unsigned long) (in valgrind-3.13.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==20303== by 0x4821D9: tex::generate_texture_patches(UniGraph const&, std::shared_ptr, mve::MeshInfo const&, std::vector >*, tex::Settings const&, std::vector >, std::allocator > > >*, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D696: main (in bin/texrecon) ==20303== ==20303== Invalid read of size 4 ==20303== at 0x548802: simple_laplacian(int, std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x549217: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== Address 0x7f65200 is 20 bytes after a block of size 108 alloc'd ==20303== at 0x4C2D1AF: operator new(unsigned long) (in valgrind-3.13.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==20303== by 0x4821D9: tex::generate_texture_patches(UniGraph const&, std::shared_ptr, mve::MeshInfo const&, std::vector >*, tex::Settings const&, std::vector >, std::allocator > > >*, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D696: main (in bin/texrecon) ==20303== ==20303== Invalid read of size 8 ==20303== at 0x5487FF: simple_laplacian(int, std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x549286: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== Address 0x7df6c68 is 12 bytes after a block of size 108 alloc'd ==20303== at 0x4C2D1AF: operator new(unsigned long) (in valgrind-3.13.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==20303== by 0x4D2E52: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== ==20303== Invalid read of size 4 ==20303== at 0x548802: simple_laplacian(int, std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x549286: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== Address 0x7df6c70 is 20 bytes after a block of size 108 alloc'd ==20303== at 0x4C2D1AF: operator new(unsigned long) (in valgrind-3.13.0/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==20303== by 0x4D2E52: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== ==20303== Invalid read of size 4 ==20303== at 0x49BF03: Eigen::SparseMatrix::sumupDuplicates() (in bin/texrecon) ==20303== by 0x49CCAA: void Eigen::internal::set_from_triplets<__gnu_cxx::__normal_iterator*, std::vector, std::allocator > > >, Eigen::SparseMatrix >(__gnu_cxx::__normal_iterator*, std::vector, std::allocator > > > const&, __gnu_cxx::__normal_iterator*, std::vector, std::allocator > > > const&, Eigen::SparseMatrix&, int) (in bin/texrecon) ==20303== by 0x549417: poisson_blend(std::shared_ptr const>, std::shared_ptr const>, std::shared_ptr >, float) (in bin/texrecon) ==20303== by 0x4DF5A3: TexturePatch::blend(std::shared_ptr const>) (in bin/texrecon) ==20303== by 0x4D35AB: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) [clone ._omp_fn.0] (in bin/texrecon) ==20303== by 0x607BDAE: GOMP_parallel (in gcc-7.3.0-lib/lib/libgomp.so.1.0.0) ==20303== by 0x4D575A: tex::local_seam_leveling(UniGraph const&, std::shared_ptr, std::vector >, std::allocator > > > const&, std::vector, std::allocator > >*) (in bin/texrecon) ==20303== by 0x44D7C5: main (in bin/texrecon) ==20303== Address 0xf932de04 is not stack'd, malloc'd or (recently) free'd ==20303== Received signal SIGSEGV (segmentation fault) Obtained 12 stack frames: 0x588f44 0x67032d0 0x49bf03 0x49ccab 0x549418 0x4df5a4 0x4d35ac 0x607bdaf 0x4d575b 0x44d7c6 0x66f0020 0x4508ca bin/texrecon(_ZN4util6system17print_stack_traceEv+0x34)[0x588f44] glibc-2.26-131/lib/libc.so.6(+0x342d0)[0x67032d0] bin/texrecon(_ZN5Eigen12SparseMatrixIfLi1EiE15sumupDuplicatesEv+0x3a3)[0x49bf03] bin/texrecon(_ZN5Eigen8internal17set_from_tripletsIN9__gnu_cxx17__normal_iteratorIPNS_7TripletIfiEESt6vectorIS5_SaIS5_EEEENS_12SparseMatrixIfLi0EiEEEEvRKT_SF_RT0_i+0x7db)[0x49ccab] bin/texrecon(_Z13poisson_blendSt10shared_ptrIKN3mve5ImageIfEEES_IKNS1_IhEEES_IS2_Ef+0x858)[0x549418] bin/texrecon(_ZN12TexturePatch5blendESt10shared_ptrIKN3mve5ImageIfEEE+0x94)[0x4df5a4] bin/texrecon[0x4d35ac] gcc-7.3.0-lib/lib/libgomp.so.1(GOMP_parallel+0x3f)[0x607bdaf] bin/texrecon(_ZN3tex19local_seam_levelingERK8UniGraphSt10shared_ptrIKN3mve12TriangleMeshEERKSt6vectorIS8_INS_20VertexProjectionInfoESaIS9_EESaISB_EEPS8_IS3_I12TexturePatchESaISH_EE+0x148b)[0x4d575b] bin/texrecon(main+0xc36)[0x44d7c6] glibc-2.26-131/lib/libc.so.6(__libc_start_main+0xf0)[0x66f0020] bin/texrecon(_start+0x2a)[0x4508ca] Segmentation fault ==20303== ==20303== HEAP SUMMARY: ==20303== in use at exit: 105,409,680 bytes in 154,282 blocks ==20303== total heap usage: 4,718,202 allocs, 4,563,920 frees, 10,446,037,939 bytes allocated ==20303== ==20303== LEAK SUMMARY: ==20303== definitely lost: 3,570,646 bytes in 61,064 blocks ==20303== indirectly lost: 1,560,816 bytes in 60,949 blocks ==20303== possibly lost: 3,648 bytes in 26 blocks ==20303== still reachable: 100,274,570 bytes in 32,243 blocks ==20303== of which reachable via heuristic: ==20303== newarray : 6,168 bytes in 3 blocks ==20303== suppressed: 0 bytes in 0 blocks ==20303== Rerun with --leak-check=full to see details of leaked memory ==20303== ==20303== For counts of detected and suppressed errors, rerun with: -v ==20303== ERROR SUMMARY: 6 errors from 6 contexts (suppressed: 0 from 0) ```

[nh2]

In a previous short segfault investigation, I made myself the following note, which might be helpful:

---

With this diff

diff --git a/libs/tex/poisson_blending.cpp b/libs/tex/poisson_blending.cpp
index 2bed1da..b0eed1e 100644
--- a/libs/tex/poisson_blending.cpp
+++ b/libs/tex/poisson_blending.cpp
@@ -34,12 +34,19 @@ bool valid_mask(mve::ByteImage::ConstPtr mask){
     const int height = mask->height();

     for (int x = 0; x < width; ++x)
-        if (mask->at(x, 0, 0) == 255 || mask->at(x, height - 1, 0) == 255)
+        if (mask->at(x, 0, 0) == 255 || mask->at(x, height - 1, 0) == 255) {
+            std::cerr << "valid_mask x failed at x=" << x << ", height=" << height << ", "
+                      << "mask->at(" << x << ", 0, 0) = " << (unsigned int) mask->at(x, 0, 0) << ", "
+                      << "mask->at(" << x << ", " << (height - 1) << ", 0) = " << (unsigned int) mask->at(x, height - 1, 0)
+                      << std::endl;
             return false;
+        }

     for (int y = 0; y < height; ++y)
-        if (mask->at(0, y, 0) == 255 || mask->at(width - 1, y, 0) == 255)
+        if (mask->at(0, y, 0) == 255 || mask->at(width - 1, y, 0) == 255) {
+            std::cerr << "valid_mask y failed" << std::endl;
             return false;
+        }

     //TODO check for sane boundary conditions...

I get:

valid_mask x failed at x=1, height=3, mask->at(1, 0, 0) = 64, mask->at(1, 2, 0) = 255
Mask check failed

[nh2]

Perhaps the 64 comes from here? https://github.com/nmoehrle/mvs-texturing/blob/3840a53de3a21c955571b6214f5b4721e11d4a97/libs/tex/texture_patch.cpp#L99

[nh2]

Using this script with gdb to print the mask in my case:

print mask->width()
print mask->height()
set $y = 0
while ($y < mask->height())
  set $x = 0
  while ($x < mask->width())
    print { (int) $x, (int) $y, 0, (int) mask->at($x, $y, 0) }
    set $x=$x+1
  end
  set $y=$y+1
  print ""
end

prints

(gdb) print mask->width()
$510 = 3
(gdb) print mask->height()
$511 = 3
(gdb) set $y = 0
(gdb) while ($y < mask->height())
 >  set $x = 0
 >  while ($x < mask->width())
  >    print { (int) $x, (int) $y, 0, (int) mask->at($x, $y, 0) }
  >    set $x=$x+1
  >  end
 >  set $y=$y+1
 >  print ""
 >end
$512 = {0, 0, 0, 64}
$513 = {1, 0, 0, 64}
$514 = {2, 0, 0, 64}
$515 = ""
$516 = {0, 1, 0, 64}
$517 = {1, 1, 0, 255}
$518 = {2, 1, 0, 255}
$519 = ""
$520 = {0, 2, 0, 64}
$521 = {1, 2, 0, 255}
$522 = {2, 2, 0, 64}
$523 = ""

So that 3x3 mask looks like:

 64  64  64
 64 255 255
 64 255  64

And valid_mask forbids the bottom middle 255 in this case.

How comes it's there?

[nh2]

In a previous short segfault investigation

Just for clarification, I have confirmed that also in this new case it is the valid_mask assertion failure:

        Blending texture patches 100%...valid_mask x failed at x=1, height=3, mask->at(1, 0, 0) = 64, mask->at(1, 2, 0) = 255
texrecon: /root/mvs-texturing/mvs-texturing-7126bb3/libs/tex/poisson_blending.cpp:64: void poisson_blend(mve::Image<float>::ConstPtr, mve::Image<unsigned char>::ConstPtr, mve::Image<float>::Ptr, float): Assertion `valid_mask(mask)' failed.

and that with assertions enabled the valgrind memory access errors disappear; so it's most likely just the assertion failure that needs to be investigated here.

[nh2]

Some insight:

I inserted after this line https://github.com/nmoehrle/mvs-texturing/blob/3840a53de3a21c955571b6214f5b4721e11d4a97/libs/tex/texture_patch.cpp#L72

                    assert(x != get_width() - 1 && y != get_height() - 1);

which triggered as well.

Again using gdb, I printed:

$1 = 0
(gdb) p min_y
$2 = 0
(gdb) p max_x
$3 = 3
(gdb) 
$4 = 3
(gdb) p max_y
$5 = 3
(gdb) p bcoords 
$6 = {v = {1, -0, 0}}
(gdb) p x
$7 = 2
(gdb) p y
$8 = 1
(gdb) p bcoords.minimum()
$9 = -0
(gdb) p tri
$10 = {v1 = {v = {2, 1}}, v2 = {v = {1, 1}}, v3 = {v = {1, 2}}, detT = -1, aabb = {min_x = 1, min_y = 1, max_x = 2, max_y = 2}}
(gdb) p v1
$11 = {v = {2, 1}}
(gdb) p v2
$12 = {v = {1, 1}}
(gdb) p v3
$13 = {v = {1, 2}}

So we have the Tri triangle {2,1}, {1,1}, {1, 2} and it computes that the coords {1, 2} are inside = true.

So mask[1][2] gets set to 255, and in general this triangle gets rendered as

 64  64  64
 64 255 255
 64 255  64

as shown above (you can see the triangle being rendered in the mask as 255s; they form a triangle).

So clearly the border area is not free of 255, as demanded by the later code.

Should it be? Should the triangle be rendered only as the pixel in the middle being 255? Also what ensures that the triangle tri doesn't have a point at {0, 0}?

[nh2]

Also interesting but probably not related to the problem: When for the Tri triangle {2,1}, {1,1}, {1, 2} it computes that the barycentric coords of {1, 2} in this triangle (should be {1, -0, 0}) are {1, -0, 0} -- negative floating point zero!

This works out OK because in the check https://github.com/nmoehrle/mvs-texturing/blob/3840a53de3a21c955571b6214f5b4721e11d4a97/libs/tex/texture_patch.cpp#L70

bool inside = bcoords.minimum() >= 0.0f; - {1, -0, 0}.minimum() will be -0.0f which is still >= 0.0f.

But I wonder whether it's intended that get_barycentric_coords() returns negative zeros.

[nh2]

For my reference, the place where the patches are generated is https://github.com/nmoehrle/mvs-texturing/blob/7126bb3aa34a4a9a36003efa06a1db5fe07509fd/libs/tex/generate_texture_patches.cpp#L114-L118; that's where the texture_patch_border = 1 is also considered (the only other place is here).

[nh2]

I've made a PR that fixes it, with full explanation, at #126.

[nmoehrle]

Thank you very much for the detailed analysis and fix! I cannot look at it at the moment but will do so within the next days.