plane segmentation does not work

[son-youngwoo]

Hi! First of all, thank you for opening up your great research. I'm trying to run plane segmentation using command below. roslaunch convex_plane_decomposition_ros convex_plane_decomposition.launch but after few seconds, It dies. I'm using Jetson AGX xavier for pc. And ubuntu 20.04, cuda 11.4, cudnn 8.4, opencv 4.5 from jetpack 5.0.2. Here is error and my pc environment. Could you tell me how to solve it? Thank you!

[son-youngwoo]

I looked up where the system died.

Problem was from code below.

planeDecompositionPipeline_ = std::make_unique<PlaneDecompositionPipeline>(config); (in ConvexPlaneDecompositionRos.cpp)

It looks like generation smart pointer, could you tell me how to solve it?

Thank you!

[rubengrandia]

What compilation flags are you using?

Could you try building in debug mode, and run the node in gdb? Maybe the stack trace can provide some further information

[son-youngwoo]

Thank you for your reply. I tried building in debug mode using command below. catkin build --cmake-args -DCMAKE_BUILD_TYPE=Debug

And I ran the node in gdb, here is the result. Is it right?

[rubengrandia]

The node didn't start in the gdb screenshot you provided. Change the launch prefix to launch-prefix="gdb -ex run --args"

After the node crashes, run the command: backtrace

[son-youngwoo]

Oh, I got it. I changed launch-prefix and ran the command backtrace.

xavier@ubuntu:~$ roslaunch convex_plane_decomposition_ros convex_plane_decomposition.launch
... logging to /home/xavier/.ros/log/2586ace0-a046-11ed-a0cb-14f6d875a29c/roslaunch-ubuntu-6684.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://ubuntu:44845/

SUMMARY
========

PARAMETERS
 * /convex_plane_decomposition_ros/contour_extraction/marginSize: 1
 * /convex_plane_decomposition_ros/elevation_topic: /elevation_mappin...
 * /convex_plane_decomposition_ros/frequency: 20.0
 * /convex_plane_decomposition_ros/height_layer: elevation
 * /convex_plane_decomposition_ros/postprocessing/extracted_planes_height_offset: 0.0
 * /convex_plane_decomposition_ros/postprocessing/nonplanar_height_offset: 0.02
 * /convex_plane_decomposition_ros/postprocessing/nonplanar_horizontal_offset: 1
 * /convex_plane_decomposition_ros/postprocessing/smoothing_box_kernel_size: 0.1
 * /convex_plane_decomposition_ros/postprocessing/smoothing_dilation_size: 0.2
 * /convex_plane_decomposition_ros/postprocessing/smoothing_gauss_kernel_size: 0.05
 * /convex_plane_decomposition_ros/preprocessing/kernelSize: 3
 * /convex_plane_decomposition_ros/preprocessing/numberOfRepeats: 1
 * /convex_plane_decomposition_ros/preprocessing/resolution: 0.04
 * /convex_plane_decomposition_ros/publish_to_controller: True
 * /convex_plane_decomposition_ros/ransac_plane_refinement/cluster_epsilon: 0.041
 * /convex_plane_decomposition_ros/ransac_plane_refinement/epsilon: 0.025
 * /convex_plane_decomposition_ros/ransac_plane_refinement/min_points: 4
 * /convex_plane_decomposition_ros/ransac_plane_refinement/normal_threshold: 25.0
 * /convex_plane_decomposition_ros/ransac_plane_refinement/probability: 0.001
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/connectivity: 4
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/global_plane_fit_angle_error_threshold_degrees: 25.0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/global_plane_fit_distance_error_threshold: 0.025
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/include_ransac_refinement: True
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/kernel_size: 3
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/local_plane_inclination_threshold_degrees: 35.0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/min_number_points_per_label: 4
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/planarity_opening_filter: 0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/plane_inclination_threshold_degrees: 30.0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/plane_patch_error_threshold: 0.02
 * /convex_plane_decomposition_ros/submap/length: 3.0
 * /convex_plane_decomposition_ros/submap/width: 3.0
 * /convex_plane_decomposition_ros/target_frame_id: odom
 * /rosdistro: noetic
 * /rosversion: 1.15.15

NODES
  /
    convex_plane_decomposition_ros (convex_plane_decomposition_ros/convex_plane_decomposition_ros_node)

auto-starting new master
process[master]: started with pid [6699]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 2586ace0-a046-11ed-a0cb-14f6d875a29c
process[rosout-1]: started with pid [6716]
started core service [/rosout]
process[convex_plane_decomposition_ros-2]: started with pid [6723]
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/xavier/catkin_ws/devel/lib/convex_plane_decomposition_ros/convex_plane_decomposition_ros_node...
Starting program: /home/xavier/catkin_ws/devel/lib/convex_plane_decomposition_ros/convex_plane_decomposition_ros_node __name:=convex_plane_decomposition_ros __log:=/home/xavier/.ros/log/2586ace0-a046-11ed-a0cb-14f6d875a29c/convex_plane_decomposition_ros-2.log
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[New Thread 0xffffeafa1300 (LWP 6729)]
[New Thread 0xffffea7a0300 (LWP 6730)]
[New Thread 0xffffe8f9f300 (LWP 6731)]
[New Thread 0xffffe679e300 (LWP 6732)]
[New Thread 0xffffe4f9d300 (LWP 6733)]
[New Thread 0xffffe379c300 (LWP 6734)]
[New Thread 0xffffe1f9b300 (LWP 6735)]
[New Thread 0xffffdff2f300 (LWP 6737)]
[New Thread 0xffffdf72e300 (LWP 6738)]
[New Thread 0xffffdef2d300 (LWP 6739)]
[New Thread 0xffffde72c300 (LWP 6740)]
[New Thread 0xffffddf2b300 (LWP 6741)]

Thread 1 "convex_plane_de" received signal SIGSEGV, Segmentation fault.
0x0000fffff6771864 in cv::getStructuringElement(int, cv::Size_<int>, cv::Point_<int>) ()
   from /lib/aarch64-linux-gnu/libopencv_imgproc.so.4.2
(gdb) backtrace
#0  0x0000fffff6771864 in cv::getStructuringElement(int, cv::Size_<int>, cv::Point_<int>) ()
   from /lib/aarch64-linux-gnu/libopencv_imgproc.so.4.2
#1  0x0000fffff6ed00d4 in convex_plane_decomposition::contour_extraction::ContourExtraction::ContourExtraction (this=0xaaaaab008a30, parameters=...)
    at /home/xavier/catkin_ws/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition/src/contour_extraction/ContourExtraction.cpp:19
#2  0x0000fffff6f583f0 in convex_plane_decomposition::PlaneDecompositionPipeline::PlaneDecompositionPipeline (this=0xaaaaab008750, config=...)
    at /home/xavier/catkin_ws/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition/src/PlaneDecompositionPipeline.cpp:10
#3  0x0000fffff7c768f4 in std::make_unique<convex_plane_decomposition::PlaneDecompositionPipeline, convex_plane_decomposition::PlaneDecompositionPipeline::Config&> ()
    at /usr/include/c++/11/bits/unique_ptr.h:962
#4  0x0000fffff7c6bec8 in convex_plane_decomposition::ConvexPlaneExtractionROS::loadParameters (
    this=0xffffffffe030, nodeHandle=...)
    at /home/xavier/catkin_ws/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition_ros/src/ConvexPlaneDecompositionRos.cpp:87
#5  0x0000fffff7c6a998 in convex_plane_decomposition::ConvexPlaneExtractionROS::ConvexPlaneExtractionROS (this=0xffffffffe030, nodeHandle=...)
    at /home/xavier/catkin_ws/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition_ros/src/ConvexPlaneDecompositionRos.cpp:18
#6  0x0000aaaaaaab2dd0 in main (argc=1, argv=0xffffffffe6e8)
    at /home/xavier/catkin_ws/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition_ros/src/ConvexPlaneDecompositionNode.cpp:15

[mujiwob]

I also encountered a problem at planeDecompositionPipeline_ = std::make_unique<PlaneDecompositionPipeline>(config);
I ran it with Ubuntu 18.04.6 LTS and ROS Melodic, my gdb output is

(elevation) cetc21@cetc21-HP-Z4-G4-Workstation:~/Downloads/3D_Reconstruction/catkin_emc$ roslaunch convex_plane_decomposition_ros convex_plane_decomposition.launch
... logging to /home/cetc21/.ros/log/e93d3678-bcb5-11ed-b508-508140251bef/roslaunch-cetc21-HP-Z4-G4-Workstation-5000.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://cetc21-HP-Z4-G4-Workstation:35401/

SUMMARY
========

PARAMETERS
 * /convex_plane_decomposition_ros/contour_extraction/marginSize: 1
 * /convex_plane_decomposition_ros/elevation_topic: /elevation_mappin...
 * /convex_plane_decomposition_ros/frequency: 20.0
 * /convex_plane_decomposition_ros/height_layer: elevation
 * /convex_plane_decomposition_ros/postprocessing/extracted_planes_height_offset: 0.0
 * /convex_plane_decomposition_ros/postprocessing/nonplanar_height_offset: 0.02
 * /convex_plane_decomposition_ros/postprocessing/nonplanar_horizontal_offset: 1
 * /convex_plane_decomposition_ros/postprocessing/smoothing_box_kernel_size: 0.1
 * /convex_plane_decomposition_ros/postprocessing/smoothing_dilation_size: 0.2
 * /convex_plane_decomposition_ros/postprocessing/smoothing_gauss_kernel_size: 0.05
 * /convex_plane_decomposition_ros/preprocessing/kernelSize: 3
 * /convex_plane_decomposition_ros/preprocessing/numberOfRepeats: 1
 * /convex_plane_decomposition_ros/preprocessing/resolution: 0.04
 * /convex_plane_decomposition_ros/publish_to_controller: True
 * /convex_plane_decomposition_ros/ransac_plane_refinement/cluster_epsilon: 0.041
 * /convex_plane_decomposition_ros/ransac_plane_refinement/epsilon: 0.025
 * /convex_plane_decomposition_ros/ransac_plane_refinement/min_points: 4
 * /convex_plane_decomposition_ros/ransac_plane_refinement/normal_threshold: 25.0
 * /convex_plane_decomposition_ros/ransac_plane_refinement/probability: 0.001
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/connectivity: 4
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/global_plane_fit_angle_error_threshold_degrees: 25.0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/global_plane_fit_distance_error_threshold: 0.025
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/include_ransac_refinement: True
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/kernel_size: 3
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/local_plane_inclination_threshold_degrees: 35.0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/min_number_points_per_label: 4
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/planarity_opening_filter: 0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/plane_inclination_threshold_degrees: 30.0
 * /convex_plane_decomposition_ros/sliding_window_plane_extractor/plane_patch_error_threshold: 0.02
 * /convex_plane_decomposition_ros/submap/length: 3.0
 * /convex_plane_decomposition_ros/submap/width: 3.0
 * /convex_plane_decomposition_ros/target_frame_id: odom
 * /rosdistro: melodic
 * /rosversion: 1.14.13

NODES
  /
    convex_plane_decomposition_ros (convex_plane_decomposition_ros/convex_plane_decomposition_ros_node)

auto-starting new master
process[master]: started with pid [5010]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to e93d3678-bcb5-11ed-b508-508140251bef
process[rosout-1]: started with pid [5021]
started core service [/rosout]
process[convex_plane_decomposition_ros-2]: started with pid [5028]
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/cetc21/Downloads/3D_Reconstruction/catkin_emc/devel/lib/convex_plane_decomposition_ros/convex_plane_decomposition_ros_node...done.
Starting program: /home/cetc21/Downloads/3D_Reconstruction/catkin_emc/devel/lib/convex_plane_decomposition_ros/convex_plane_decomposition_ros_node __name:=convex_plane_decomposition_ros __log:=/home/cetc21/.ros/log/e93d3678-bcb5-11ed-b508-508140251bef/convex_plane_decomposition_ros-2.log
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[tcsetpgrp failed in terminal_inferior: 无效的参数]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[tcsetpgrp failed in terminal_inferior: 无效的参数]
[tcsetpgrp failed in terminal_inferior: 无效的参数]
[New Thread 0x7fffd607c700 (LWP 5035)]
[New Thread 0x7fffd587b700 (LWP 5036)]
[New Thread 0x7fffd507a700 (LWP 5037)]
[New Thread 0x7fffd4879700 (LWP 5038)]
[New Thread 0x7fffc7fff700 (LWP 5039)]

Thread 1 "convex_plane_de" received signal SIGSEGV, Segmentation fault.
0x00007ffff0be767c in cv::getStructuringElement(int, cv::Size_<int>, cv::Point_<int>) () from /usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.3.2
(gdb) backtrace 
#0  0x00007ffff0be767c in cv::getStructuringElement(int, cv::Size_<int>, cv::Point_<int>) () from /usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.3.2
#1  0x00007ffff41f3fd7 in convex_plane_decomposition::contour_extraction::ContourExtraction::ContourExtraction (this=0x555555cca3d0, parameters=...)
    at /home/cetc21/Downloads/3D_Reconstruction/catkin_emc/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition/src/contour_extraction/ContourExtraction.cpp:19
#2  0x00007ffff4277357 in convex_plane_decomposition::PlaneDecompositionPipeline::PlaneDecompositionPipeline (this=0x555555cca0f0, config=...)
    at /home/cetc21/Downloads/3D_Reconstruction/catkin_emc/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition/src/PlaneDecompositionPipeline.cpp:10
#3  0x00007ffff716b1d5 in std::make_unique<convex_plane_decomposition::PlaneDecompositionPipeline, convex_plane_decomposition::PlaneDecompositionPipeline::Config&> () at /usr/include/c++/11/bits/unique_ptr.h:962
#4  0x00007ffff7162264 in convex_plane_decomposition::ConvexPlaneExtractionROS::loadParameters (this=0x7fffffffbfc0, nodeHandle=...)
    at /home/cetc21/Downloads/3D_Reconstruction/catkin_emc/src/elevation_mapping_cupy/plane_segmentation/convex_plane_decomposition_ros/src/ConvexPlaneDecompositionRos.cpp:87
#5  0x00007ffff716072c in convex_plane_decomposition::ConvexPlaneExtractionROS::ConvexPlaneExtractionROS (this=0x7fffffffbfc0, nodeHandle=...)
    at /home/cetc21/Downloads/3D_Reconstruction/catkin_emc/src/elevation_mapping---Type <return> to continue, or q <return> to quit---

[mujiwob]

I found how to solve this problem. It is something wrong with apt installed opencv by ros (for noetic, it is opencv4.2) on jetson platform. I am not so familiar with gdb, but from the backtrace output, I think the problem comes from libopencv_imgproc.so.4.2. I tried to set opencv version in CMakeLists.txt by find_package(OpenCV 4.6.0 REQUIRED) (I installed opencv4.6 from source), but the problem also occurred. Therefore, I thought the problem came from other ros packages which based on opencv4.2, for example, grid-map and cv-bridge. I don't know if there is a better way to solve this problem, but I have found two solutions (maybe one and a half?). Solution 1. Uninstall libopencv4.2, and build opencv4.2 based ros packages from source.

sudo apt purge libopencv*4.2
(apt will list all libopencv-xxx4.2 and ros packages based on it)
mkdir -p dependencies/src && cd dependencies/src
git clone ros packages above
cd ..
catkin build --cmake-args -DCMAKE_BUILD_TYPE=Release

Then, you can build elevation_mapping_cupy by instruction. It works for me.

Solution 2. Replace opencv4.2 with another version of opencv.

cd /lib/aarch64-linux-gnu
sudo mv libopencv_imgproc.so.4.2 libopencv_imgproc.so.4.2.bak
sudo ln -s libopencv_imgproc.so.4.5 libopencv_imgproc.so.4.2

This can solve the problem caused by libopencv_imgproc.so.4.2, but I encountered a new problem with libopencv_core.so.4.2. Maybe it can be solved like libopencv_imgproc.so.4.2, but I didn't try.

I don't think either of these two methods are optimal because they may cause other problems that I am not aware of, but they do solve the problem at hand.I would be grateful If anyone can suggest a better solution.

[hsurya08]

I found how to solve this problem. It is something wrong with apt installed opencv by ros (for noetic, it is opencv4.2) on jetson platform. I am not so familiar with gdb, but from the backtrace output, I think the problem comes from libopencv_imgproc.so.4.2. I tried to set opencv version in CMakeLists.txt by find_package(OpenCV 4.6.0 REQUIRED) (I installed opencv4.6 from source), but the problem also occurred. Therefore, I thought the problem came from other ros packages which based on opencv4.2, for example, grid-map and cv-bridge. I don't know if there is a better way to solve this problem, but I have found two solutions (maybe one and a half?). Solution 1. Uninstall libopencv4.2, and build opencv4.2 based ros packages from source.

sudo apt purge libopencv*4.2
(apt will list all libopencv-xxx4.2 and ros packages based on it)
mkdir -p dependencies/src && cd dependencies/src
git clone ros packages above
cd ..
catkin build --cmake-args -DCMAKE_BUILD_TYPE=Release

Then, you can build elevation_mapping_cupy by instruction. It works for me.

Solution 2. Replace opencv4.2 with another version of opencv.

cd /lib/aarch64-linux-gnu
sudo mv libopencv_imgproc.so.4.2 libopencv_imgproc.so.4.2.bak
sudo ln -s libopencv_imgproc.so.4.5 libopencv_imgproc.so.4.2

This can solve the problem caused by libopencv_imgproc.so.4.2, but I encountered a new problem with libopencv_core.so.4.2. Maybe it can be solved like libopencv_imgproc.so.4.2, but I didn't try.

I don't think either of these two methods are optimal because they may cause other problems that I am not aware of, but they do solve the problem at hand.I would be grateful If anyone can suggest a better solution.

Hi @woblitent, Thank you for the detailed instructions. I was actually facing the similar issue and was able to solve the error by following the solution-1 that you have explained. However, when I installed the grid_map using git clone it was giving me another similar runtime error for the elevation_mapping node. I have attached the screenshot also. So in brief, when I install grid_map using "apt install ..." command, I am getting error for the convex_plane_decomposition_ros node and when I install grid_map using git clone I am getting error for the elevation_mapping node. I was not able to find a way to run both without any errors. I guess the error is due to grid_map_filters. Please do let me know if you have any suggestions on how to solve this issue. Any information that you provide would be really helpful. Thanks!

[lucasbrodo]

Hi @hsurya08; Have you resolved your problem ? I'm facing the same issues. Thanks for your help !