atenpas / gpd_ros

ROS wrapper around GPD
BSD 2-Clause "Simplified" License
57 stars 70 forks source link

[detect_grasps-1] process has died [pid 26762, exit code -11, #12

Open jediofgever opened 4 years ago

jediofgever commented 4 years ago

Hello, I am trying to use this lib with my simulated point cloud under gazebo,but I am getting error in title. I see the log but there is no any meaningful output for further debug. Here is what happens when Launch the node;


atas@atas-G7-7590:~/catkin_ws$ roslaunch gpd_ros ur5.launch 
... logging to /home/atas/.ros/log/9273b2be-62b2-11ea-ad7d-e4b97aeb78f3/roslaunch-atas-G7-7590-26747.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
WARNING: disk usage in log directory [/home/atas/.ros/log] is over 1GB.
It's recommended that you use the 'rosclean' command.

started roslaunch server http://atas-G7-7590:36443/

SUMMARY
========

PARAMETERS
 * /detect_grasps/cloud_topic: /camera/depth/points
 * /detect_grasps/cloud_type: 0
 * /detect_grasps/config_file: /home/atas/gpd/cf...
 * /detect_grasps/rviz_topic: plot_grasps
 * /detect_grasps/samples_topic: 
 * /rosdistro: melodic
 * /rosversion: 1.14.3

NODES
  /
    detect_grasps (gpd_ros/detect_grasps)

ROS_MASTER_URI=http://localhost:11311

process[detect_grasps-1]: started with pid [26762]
Init ....
============ HAND GEOMETRY ======================
finger_width: 0.01
hand_outer_diameter: 0.12
hand_depth: 0.06
hand_height: 0.02
init_bite: 0.01
=================================================
============ PLOTTING ========================
plot_normals: false
plot_samples false
plot_candidates: false
plot_filtered_candidates: false
plot_valid_grasps: false
plot_clustered_grasps: false
plot_selected_grasps: true
==============================================
============ CLOUD PREPROCESSING =============
voxelize: true
voxel_size: 0.003
remove_outliers: false
workspace: -1.00 1.00 -1.00 1.00 -1.00 1.00 
sample_above_plane: false
normals_radius: 0.030
refine_normals_k: 0
==============================================
============ CANDIDATE GENERATION ============
num_samples: 500
num_threads: 4
nn_radius: 0.01
hand axes: 2 
num_orientations: 8
num_finger_placements: 10
deepen_hand: true
friction_coeff: 20.00
min_viable: 6
==============================================
============ GRASP IMAGE GEOMETRY ===============
volume width: 0.1
volume depth: 0.06
volume height: 0.02
image_size: 60
image_num_channels: 15
=================================================
NET SETUP runtime: 0.12472
============ CLASSIFIER ======================
model_file: 
weights_file: /home/atas/gpd/models/lenet/15channels/params/
batch_size: 1
==============================================
============ CANDIDATE FILTERING =============
candidate_workspace: -1.00 1.00 -1.00 1.00 -1.00 1.00 
min_aperture: 0.0000
max_aperture: 0.8500
==============================================
============ CLUSTERING ======================
min_inliers: 1
==============================================

Created GPD ....
[ INFO] [1583834549.041823993]: Waiting for point cloud to arrive ...
[ INFO] [1583834549.278995343, 4695.919000000]: Received cloud with 307200 points.
Processing cloud with 307200 points.
Voxelized cloud: 113000
Calculating surface normals ...
num_threads: 4
 runtime(computeNormals): 2.4907
camera: 0, #indices: 113000, #normals: 113000 
Calculated 113000 surface normals in 2.5002s (mode: OpenMP).
Reversing direction of normals that do not point to at least one camera ...
 reversed 0 normals
 runtime (reverse normals): 0.00284553
Estimating local reference frames ...
Estimated 500 frames in 0.0042s.
Finding hand poses ...
[detect_grasps-1] process has died [pid 26762, exit code -11, cmd /home/atas/catkin_ws/devel/lib/gpd_ros/detect_grasps __name:=detect_grasps __log:=/home/atas/.ros/log/9273b2be-62b2-11ea-ad7d-e4b97aeb78f3/detect_grasps-1.log].
log file: /home/atas/.ros/log/9273b2be-62b2-11ea-ad7d-e4b97aeb78f3/detect_grasps-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done

is there any suggestions ? Thank you for your time

seann999 commented 4 years ago

Same here

Console:

Created GPD ....
[ INFO] [1584601310.962219117]: Waiting for point cloud to arrive ...
[ INFO] [1584601338.677122170]: Received cloud with 307200 points.
Processing cloud with 307200 points.
Cloud after removing NANs: 269020
[detect_grasps-1] process has died [pid 18329, exit code -11, cmd /root/ROB/catkin_ws/devel/lib/gpd_ros/detect_grasps __name:=detect_grasps __log:=/root/.ros/log/20200319-053710_add2b6a2-69a3-11ea-adb8-0030644a53eb/detect_grasps-1.log].
log file: /root/.ros/log/20200319-053710_add2b6a2-69a3-11ea-adb8-0030644a53eb/detect_grasps-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done
seann999 commented 4 years ago

I think I solved it after following a comment here

FrankLin9981 commented 4 years ago

@seann999 Excuse me. Could you provide some details about how you solve this problem? Thanks.

jediofgever commented 4 years ago

@seann999 the issue you point is related to PCL, however the output of gdb seems like error is triggered by Eigen;


Thread 1 "detect_grasps" received signal SIGSEGV, Segmentation fault.
Eigen::internal::dense_assignment_loop<Eigen::internal::generic_dense_assignment_kernel<Eigen::internal::evaluator<Eigen::Matrix<float, -1, -1, 0, -1, -1> >, Eigen::internal::evaluator<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1> const, 0, Eigen::Stride<0, 0> > >, Eigen::internal::assign_op<float, float>, 0>, 0, 0>::run (kernel=<synthetic pointer>...)
    at /usr/include/eigen3/Eigen/src/Core/AssignEvaluator.h:326
326         kernel.assignCoeffByOuterInner(outer, inner);
(gdb) BT
jediofgever commented 4 years ago

This issue happened to me because the network paths were not correctly found

I used Eigen classifier, which uses some parameters under lenet folder. I think @atenpas moved lenet folder from the root of project to models folder but the default path in the ros_eigen_params.cfg was not updated accordingly. Finally leading this issue. So anyone who met this issue make sure your path to the classifiers are correctly set

darmin-hsiung commented 3 years ago

The same question has show when I run $ roslaunch gpd tutorial1.launch This gpd code is forword version, please help me find where the question at? Thanks again. The error log are as follows

process[detect_grasps-1]: started with pid [9269] NET SETUP runtime: 0.0985822 [ INFO] [1614949591.731973322]: Waiting for point cloud to arrive ... [ INFO] [1614949591.976138311]: Received cloud with 170258 points. Processing cloud with: 170258 points. After workspace filtering: 170258 points left. double free or corruption (out) [detect_grasps-1] process has died [pid 9269, exit code -6, cmd /home/wyu/catkin_ws/devel/lib/gpd/detect_grasps __name:=detect_grasps __log:=/home/wyu/.ros/log/c8eacd48-7db2-11eb-9f63-0024d7d351e8/detect_grasps-1.log]. log file: /home/wyu/.ros/log/c8eacd48-7db2-11eb-9f63-0024d7d351e8/detect_grasps-1*.log all processes on machine have died, roslaunch will exit shutting down processing monitor... ... shutting down processing monitor complete done

FabianSchuetze commented 3 years ago

I was just wrapping a unique pointer around the cloud_camera object in the grasp_detection_node.cpp file. Deleting this object causes the segfault on my machine.

zhaoguangbao commented 3 years ago

[detect_grasps-1] process has died Make sure the path to the .cfg file is set correctly before running roslaunch. for example

# Path to config file for robot hand geometry
hand_geometry_filename = /home/zhao/git_ws/gpd/cfg/hand_geometry.cfg

# Path to config file for volume and image geometry
image_geometry_filename = /home/zhao/git_ws/gpd/cfg/image_geometry_15channels.cfg

# Path to directory that contains neural network parameters
weights_file = /home/zhao/git_ws/gpd/models/lenet/15channels/params/

Maybe "double free or corruption (out)" error will occur, which can be solved as follows

// grasp_detection_node.cpp
void GraspDetectionNode::cloud_callback(const sensor_msgs::PointCloud2& msg)
{
  if (!has_cloud_)
  {
      if(cloud_camera_)
        delete cloud_camera_;
      cloud_camera_ = NULL;
      // ...
friolero commented 2 years ago

[detect_grasps-1] process has died Make sure the path to the .cfg file is set correctly before running roslaunch. for example

# Path to config file for robot hand geometry
hand_geometry_filename = /home/zhao/git_ws/gpd/cfg/hand_geometry.cfg

# Path to config file for volume and image geometry
image_geometry_filename = /home/zhao/git_ws/gpd/cfg/image_geometry_15channels.cfg

# Path to directory that contains neural network parameters
weights_file = /home/zhao/git_ws/gpd/models/lenet/15channels/params/

Maybe "double free or corruption (out)" error will occur, which can be solved as follows

// grasp_detection_node.cpp
void GraspDetectionNode::cloud_callback(const sensor_msgs::PointCloud2& msg)
{
  if (!has_cloud_)
  {
      if(!cloud_camera_)
        delete cloud_camera_;
      cloud_camera_ = NULL;
      // ...

I'm experincing memory leak with adding the if (!cloudcamera) for the 'double free or corruption (out)' error. Please be careful if anyone is doing that.

Removing the '-march=native -msse4.2 -mavx2 -mfma -flto' flags in gpd CMakeLists and rebuilding gpd + gpd_ros somehow solves the issue for me. Appreciate if anyone has more insight on this issue and why this helps (or hides the problem if it is?)

zhaoguangbao commented 1 year ago

For memory leak, updating if(!cloud_camera_) to if(cloud_camera_) may be helpful.

hqf2591102423 commented 10 months ago

我遇到这个问题是因为未正确找到网络路径

我使用了特征分类器,它使用lenet文件夹下的一些参数。我认为@atenpaslenet将文件夹从项目根目录 移动到models文件夹,但默认路径ros_eigen_params.cfg未相应更新。终于主导了这个问题。 因此,遇到此问题的任何人都请确保正确设置了分类器的路径

确实是这个问题

zhanghua7099 commented 1 month ago

Hi! Does anyone have a similar problem in Ubuntu 20.04?

....
==============================================
============ CLUSTERING ======================
min_inliers: 1
==============================================

Created GPD ....
[ INFO] [1729247172.602160208]: Waiting for point cloud to arrive ...
[ INFO] [1729247178.467983706, 2216.431000000]: Received cloud with 278528 points.
Processing cloud with 278528 points.
[detect_grasps-1] process has died [pid 172968, exit code -11, cmd /home/aa/ROSProject/gpd_ws/devel/lib/gpd_ros/detect_grasps __name:=detect_grasps __log:=/home/aa/.ros/log/38559f6e-8d36-11ef-93d5-e15d887eb9da/detect_grasps-1.log].
log file: /home/aa/.ros/log/38559f6e-8d36-11ef-93d5-e15d887eb9da/detect_grasps-1*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done

It seems that after gpd receives the pcd from gpd_ros, it will be dead...

Directly using gpd will be ok:

cd ~/gpd/build
./detect_grasps ../cfg/ros_eigen_params.cfg ../tutorials/krylon.pcd

How to solve this problem? I really need your help......

atenpas commented 1 month ago

@zhanghua7099 Could you put the point cloud that you tried with gpd_ros into a pcd file and try it with detect_grasps? Or upload it so I can have a look?

zhanghua7099 commented 1 month ago

@atenpas Hi! Thanks for your reply. I have fixed this problem. In my computer (Ubuntu 20.04), this is caused by the different compile setting of gpd and gpd_ros.

In gpd: https://github.com/atenpas/gpd/blob/6327f20eabfcba41a05fdd2e2ba408153dc2e958/CMakeLists.txt#L29

However, in gpd_ros: https://github.com/atenpas/gpd_ros/blob/9ae524193acecd465ef324c1f3c621a38ae9c31d/CMakeLists.txt#L28

I installed gpd via mkdir build && cd build && cmake .. && make -j && sudo make install.

It seems that gpd_ros will directly use the intalled gpd. If these two compile options are different, gpd_ros will fail.

I set the compile options for the two libraries to be the same, and the results are good.