[running error] segmentation fault (invalid read) when running crophull

[EmilyJrxx]

Describe the bug

Segmentation fault(core dumped) when running crophull (with a 3d convex hull), and code itself is almost the same with example code given within some blogs like https://blog.csdn.net/zfjBIT/article/details/92795689 (forgive me, i know this piece of example has been reprinted many times in a bunch of websites). I run the code with Valgrind and some 'invalid read of size 4' were reported.

Context I expected to crop a part out of a scene cloud with convex_hull and crop_hull, yet a Segmentation fault(core dumped) was reported.

To Reproduce This is the code i used to reproduce the error, it's actually a step of a larger pipeline, the bounding box was supposed to generate via 2D detection and projection, Here i used fixed value from the pipeline instead.

As you can see, i intended to crop the cloud with a pyramid-like convex hull.

#include <iostream>
#include <pcl/io/ply_io.h>
#include <pcl/point_cloud.h>
#include <pcl/point_types.h>
#include <pcl/filters/crop_hull.h>
#include <pcl/surface/concave_hull.h>
#include <pcl/surface/convex_hull.h>

using namespace pcl;
using namespace std;

int main(int argc, char** argv)
{
    string cloud_file = argv[1];
    PointCloud<PointXYZ>::Ptr cloud (new PointCloud<PointXYZ>);
    io::loadPLYFile(cloud_file, *cloud);

    PointXYZ left_top, left_bot, right_top, right_bot;
    left_top.x = -0.3885; left_top.y = -0.5662; left_top.z = 1.4197;
    left_bot.x = -0.3885; left_bot.y = 0.0485; left_bot.z = 1.4197;
    right_top.x = -0.0396; right_top.y = -0.5662; right_top.z = 1.4197;
    right_bot.x = -0.0396; right_bot.y = 0.0485; right_bot.z = 1.4197;

    left_top.z  += 0.15;
    left_bot.z  += 0.15;
    right_top.z += 0.15;
    right_bot.z += 0.15;

    PointCloud<PointXYZ>::Ptr boundingbox_3d (new PointCloud<PointXYZ>);
    boundingbox_3d -> push_back(left_top);
    boundingbox_3d -> push_back(left_bot);
    boundingbox_3d -> push_back(right_top);
    boundingbox_3d -> push_back(right_bot);
    boundingbox_3d -> push_back(PointXYZ(0.0, 0.0, 0.0));

    cout << "creating hull" << endl; //debug
    ConvexHull<PointXYZ> hull;
    hull.setInputCloud(boundingbox_3d);
    hull.setDimension(3);
    vector<Vertices> polygons;
    PointCloud<PointXYZ>::Ptr surface_hull (new PointCloud<PointXYZ>);
    cout << "reconstructing hull" << endl; //debug
    hull.reconstruct(*surface_hull, polygons);
    cout << "creating crophull" << endl; //debug

    PointCloud<PointXYZ>::Ptr cropped (new PointCloud<PointXYZ>);
    CropHull<PointXYZ> bb_filter; 
    // bb_filter.setDim(3);
    bb_filter.setInputCloud(cloud);
    bb_filter.setHullIndices(polygons);
    bb_filter.setHullCloud(surface_hull);
    cout << "cropping cloud" << endl;
    bb_filter.filter(*cropped);

    printf("Cropping : %d / %d \n", cloud->size(), cropped->size());
    return 0;
}

Screenshots/Code snippets

GDB bug report Valgrind bug report

In order to help explain your problem, please consider adding

• screenshots of the GUI issues
• code snippets: syntax for code with correct language highlights

Your Environment (please complete the following information):

• OS: Ubuntu 18.04
• Compiler: gcc 7.5.0
• PCL Version: 1.9.1

Possible Solution

I tried to rebuild PCL library with Eigen3.3.4 (3.3.7 originally), or use PCL 1.11.0 instead, still cannot resolve this error.

Additional context

What confused me is that, i ran the exact same code in another laptop, with almost the same environment configuration (Os, Complier, PCL Version, Eigen3 Version), and no Segmentation fault nor errors from Valgrind, so I guess maybe it's not due to PCL repository bug, but something wrong during the building of PCL library or code compilation.

I don't know if this can be counted as a PCL bug (probably not), yet much appreciate it if you can give some advice.

[mvieth]

Could be that there is first an error in ConvexHull, which then produces the error in CropHull. My best guess is that at the line you see in the gdb log, the cloud is access with an invalid index. In your second screenshot, there is some information cut off at the right side. Could you post that log again as a txt? Could you also check polygons, to see if there are any invalid indices in the Vertices? Please also set a good title for this issue.

[EmilyJrxx]

@mvieth Thanks for your quick reply! I further looked into surface_hull and polygons and actually found something wrong with the hull.reconstruct() function: As you can see in my code, the bounding box was supposed to form a rectangular pyramid hull.

I also ran the exact code on another laptop (mentioned in original issue description, which reported no error in almost the same environment configurations), and size of surface hull is 5, and size of polygons is 6, which should be correct.

The complete Error Summary mentioned above is as follows:

==30180== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 0 from 0) ==30180== ==30180== 1 errors in context 1 of 2: ==30180== Invalid read of size 4 ==30180== at 0x5DD595F: coeff (CoreEvaluators.h:834) ==30180== by 0x5DD595F: assignCoeff (AssignEvaluator.h:631) ==30180== by 0x5DD595F: assignCoeffByOuterInner (AssignEvaluator.h:645) ==30180== by 0x5DD595F: run (AssignEvaluator.h:206) ==30180== by 0x5DD595F: run (AssignEvaluator.h:435) ==30180== by 0x5DD595F: call_dense_assignment_loop<Eigen::Matrix<float, 3, 1>, Eigen::Map<const Eigen::Matrix<float, 3, 1> >, Eigen::internal::assign_op<float, float> > (AssignEvaluator.h:741) ==30180== by 0x5DD595F: run (AssignEvaluator.h:879) ==30180== by 0x5DD595F: call_assignment_no_alias<Eigen::Matrix<float, 3, 1>, Eigen::Map<const Eigen::Matrix<float, 3, 1> >, Eigen::internal::assign_op<float, float> > (AssignEvaluator.h:836) ==30180== by 0x5DD595F: _set_noalias<Eigen::Map<const Eigen::Matrix<float, 3, 1> > > (PlainObjectBase.h:732) ==30180== by 0x5DD595F: PlainObjectBase<Eigen::Map<const Eigen::Matrix<float, 3, 1> > > (PlainObjectBase.h:537) ==30180== by 0x5DD595F: Matrix<Eigen::Map<const Eigen::Matrix<float, 3, 1> > > (Matrix.h:377) ==30180== by 0x5DD595F: pcl::CropHull::rayTriangleIntersect(pcl::PointXYZ const&, Eigen::Matrix<float, 3, 1, 0, 3, 1> const&, pcl::Vertices const&, pcl::PointCloud const&) (crop_hull.hpp:296) ==30180== by 0x5DD5DCB: pcl::CropHull::applyFilter3D(pcl::PointCloud&) (crop_hull.hpp:200) ==30180== by 0x5DD5F61: pcl::CropHull::applyFilter(pcl::PointCloud&) (crop_hull.hpp:64) ==30180== by 0x11F7D7: pcl::Filter::filter(pcl::PointCloud&) (filter.h:143) ==30180== by 0x119D90: main (crophull_test.cpp:53) ==30180== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==30180== ==30180== ==30180== 4 errors in context 2 of 2: ==30180== Invalid read of size 4 ==30180== at 0x7B75674: pcl::ConvexHull::performReconstruction3D(pcl::PointCloud&, std::vector<pcl::Vertices, std::allocator >&, bool) (convex_hull.hpp:402) ==30180== by 0x7B7D18A: pcl::ConvexHull::performReconstruction(pcl::PointCloud&, std::vector<pcl::Vertices, std::allocator >&, bool) (convex_hull.hpp:429) ==30180== by 0x7B7D233: pcl::ConvexHull::reconstruct(pcl::PointCloud&, std::vector<pcl::Vertices, std::allocator >&) (convex_hull.hpp:489) ==30180== by 0x119C4B: main (crophull_test.cpp:43) ==30180== Address 0x152a0c34 is 16 bytes after a block of size 4 alloc'd ==30180== at 0x4C3052A: operator new(unsigned long) (vg_replace_malloc.c:342) ==30180== by 0x5CEEAE4: allocate (new_allocator.h:111) ==30180== by 0x5CEEAE4: allocate (alloc_traits.h:436) ==30180== by 0x5CEEAE4: _M_allocate (stl_vector.h:172) ==30180== by 0x5CEEAE4: _M_create_storage (stl_vector.h:187) ==30180== by 0x5CEEAE4: _Vector_base (stl_vector.h:138) ==30180== by 0x5CEEAE4: std::vector<int, std::allocator >::vector(unsigned long, std::allocator const&) (stl_vector.h:284) ==30180== by 0x7B754CF: pcl::ConvexHull::performReconstruction3D(pcl::PointCloud&, std::vector<pcl::Vertices, std::allocator >&, bool) (convex_hull.hpp:366) ==30180== by 0x7B7D18A: pcl::ConvexHull::performReconstruction(pcl::PointCloud&, std::vector<pcl::Vertices, std::allocator >&, bool) (convex_hull.hpp:429) ==30180== by 0x7B7D233: pcl::ConvexHull::reconstruct(pcl::PointCloud&, std::vector<pcl::Vertices, std::allocator >&) (convex_hull.hpp:489) ==30180== by 0x119C4B: main (crophull_test.cpp:43) ==30180== ==30180== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 0 from 0)

[mvieth]

Which QHULL version do you have installed and how did you install it?

[EmilyJrxx]

Is version 7.2.0 (2015.2) the correct version ?

Originally I had installed libqhull-2015.2-4 (7.2.0) through debian packages: sudo apt-get install libqhull* sudo apt-get install libgtest-dev

sudo apt search libqhull* [sudo] password for xxwang: Sorting... Done Full Text Search... Done libqhull-dev/bionic,now 2015.2-4 amd64 [installed] calculate convex hulls and related structures (development files)

libqhull-doc/bionic,bionic,now 2015.2-4 all [installed] calculate convex hulls and related structures (documentation files)

libqhull-r7/bionic,now 2015.2-4 amd64 [installed] calculate convex hulls and related structures (reentrant shared library)

libqhull7/bionic,now 2015.2-4 amd64 [installed] calculate convex hulls and related structures (shared library)

I tried buiding QHULL from source this morning,and configuring qhull-related path in PCL building procedure. I tried two source packages below, but both cannot resolve the issue: https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/qhull/2015.2-4/qhull_2015.2.orig.tar.gz https://github.com/qhull/qhull/archive/v7.2.0.tar.gz

[EmilyJrxx]

I think I figured out what happened here. Originally besides debian-built QHULL package, I also installed QHULL-2020.1 from source. Although 2020.1 is installed into /usr/local/ directory, somehow these two versions 'mixed up' when building PCL library. I found this when I reinstalled debian-built QHULL and rebuild PCL, a CMake Warning told me that includes and libs of QHULL in /usr/local/ may 'collide' with debian-built ones, maybe it gave the same warinings before which I didn't notice. So problem solved after I made sure all QHULL resources built from source uninstalled ,reinstalled QHULL with debian packages, and rebuilt PCL. Thanks for your tips and quick reply. @mvieth