[detect_grasps-1] process has died

[ojijinWang]

I am a master student in Kyoto University, Japan. Thanks for your code. I have build up the code successfully. and run the tutorial0 successfully. But when i try to launch the tutorial1, the process died.

I use realsense camera D435 as the depth camera. it generates a colored pointcloud2 with 640x480 points at 30Hz. I use the ros, Kinetic. I use the caffe, CPU_ONLY, with opencv3.3

I am not familiar with your code. So I need your help. How should i fix this? or should i change some params of the depth camera(so many points?)?

Thanks

process[detect_grasps-1]: started with pid [11863] WARNING: Logging before InitGoogleLogging() is written to STDERR I0720 12:28:23.879094 11863 net.cpp:51] Initializing net from parameters: name: "LeNet" state { phase: TEST level: 0 } layer { name: "data" type: "MemoryData" top: "data" top: "label" memory_data_param { batch_size: 100 channels: 15 height: 60 width: 60 } } layer { name: "conv1" type: "Convolution" bottom: "data" top: "conv1" convolution_param { num_output: 20 kernel_size: 5 weight_filler { type: "xavier" } } } layer { name: "pool1" type: "Pooling" bottom: "conv1" top: "pool1" pooling_param { pool: MAX kernel_size: 2 stride: 2 } } layer { name: "conv2" type: "Convolution" bottom: "pool1" top: "conv2" convolution_param { num_output: 50 kernel_size: 5 weight_filler { type: "xavier" } } } layer { name: "pool2" type: "Pooling" bottom: "conv2" top: "pool2" pooling_param { pool: MAX kernel_size: 2 stride: 2 } } layer { name: "ip1" type: "InnerProduct" bottom: "pool2" top: "ip1" inner_product_param { num_output: 500 weight_filler { type: "xavier" } } } layer { name: "relu1" type: "ReLU" bottom: "ip1" top: "ip1" } layer { name: "ip2" type: "InnerProduct" bottom: "ip1" top: "ip2" inner_product_param { num_output: 2 weight_filler { type: "xavier" } } } I0720 12:28:23.879302 11863 layer_factory.hpp:77] Creating layer data I0720 12:28:23.879349 11863 net.cpp:84] Creating Layer data I0720 12:28:23.879359 11863 net.cpp:380] data -> data I0720 12:28:23.879405 11863 net.cpp:380] data -> label I0720 12:28:23.879451 11863 net.cpp:122] Setting up data I0720 12:28:23.879465 11863 net.cpp:129] Top shape: 100 15 60 60 (5400000) I0720 12:28:23.879470 11863 net.cpp:129] Top shape: 100 (100) I0720 12:28:23.879473 11863 net.cpp:137] Memory required for data: 21600400 I0720 12:28:23.879482 11863 layer_factory.hpp:77] Creating layer conv1 I0720 12:28:23.879503 11863 net.cpp:84] Creating Layer conv1 I0720 12:28:23.879510 11863 net.cpp:406] conv1 <- data I0720 12:28:23.879520 11863 net.cpp:380] conv1 -> conv1 I0720 12:28:23.879657 11863 net.cpp:122] Setting up conv1 I0720 12:28:23.879663 11863 net.cpp:129] Top shape: 100 20 56 56 (6272000) I0720 12:28:23.879667 11863 net.cpp:137] Memory required for data: 46688400 I0720 12:28:23.879685 11863 layer_factory.hpp:77] Creating layer pool1 I0720 12:28:23.879691 11863 net.cpp:84] Creating Layer pool1 I0720 12:28:23.879695 11863 net.cpp:406] pool1 <- conv1 I0720 12:28:23.879700 11863 net.cpp:380] pool1 -> pool1 I0720 12:28:23.879722 11863 net.cpp:122] Setting up pool1 I0720 12:28:23.879727 11863 net.cpp:129] Top shape: 100 20 28 28 (1568000) I0720 12:28:23.879730 11863 net.cpp:137] Memory required for data: 52960400 I0720 12:28:23.879734 11863 layer_factory.hpp:77] Creating layer conv2 I0720 12:28:23.879739 11863 net.cpp:84] Creating Layer conv2 I0720 12:28:23.879741 11863 net.cpp:406] conv2 <- pool1 I0720 12:28:23.879745 11863 net.cpp:380] conv2 -> conv2 I0720 12:28:23.879889 11863 net.cpp:122] Setting up conv2 I0720 12:28:23.879894 11863 net.cpp:129] Top shape: 100 50 24 24 (2880000) I0720 12:28:23.879896 11863 net.cpp:137] Memory required for data: 64480400 I0720 12:28:23.879901 11863 layer_factory.hpp:77] Creating layer pool2 I0720 12:28:23.879905 11863 net.cpp:84] Creating Layer pool2 I0720 12:28:23.879909 11863 net.cpp:406] pool2 <- conv2 I0720 12:28:23.879914 11863 net.cpp:380] pool2 -> pool2 I0720 12:28:23.879918 11863 net.cpp:122] Setting up pool2 I0720 12:28:23.879922 11863 net.cpp:129] Top shape: 100 50 12 12 (720000) I0720 12:28:23.879925 11863 net.cpp:137] Memory required for data: 67360400 I0720 12:28:23.879928 11863 layer_factory.hpp:77] Creating layer ip1 I0720 12:28:23.879933 11863 net.cpp:84] Creating Layer ip1 I0720 12:28:23.879936 11863 net.cpp:406] ip1 <- pool2 I0720 12:28:23.879940 11863 net.cpp:380] ip1 -> ip1 I0720 12:28:23.902318 11863 net.cpp:122] Setting up ip1 I0720 12:28:23.902357 11863 net.cpp:129] Top shape: 100 500 (50000) I0720 12:28:23.902361 11863 net.cpp:137] Memory required for data: 67560400 I0720 12:28:23.902374 11863 layer_factory.hpp:77] Creating layer relu1 I0720 12:28:23.902382 11863 net.cpp:84] Creating Layer relu1 I0720 12:28:23.902386 11863 net.cpp:406] relu1 <- ip1 I0720 12:28:23.902392 11863 net.cpp:367] relu1 -> ip1 (in-place) I0720 12:28:23.902417 11863 net.cpp:122] Setting up relu1 I0720 12:28:23.902421 11863 net.cpp:129] Top shape: 100 500 (50000) I0720 12:28:23.902436 11863 net.cpp:137] Memory required for data: 67760400 I0720 12:28:23.902438 11863 layer_factory.hpp:77] Creating layer ip2 I0720 12:28:23.902443 11863 net.cpp:84] Creating Layer ip2 I0720 12:28:23.902464 11863 net.cpp:406] ip2 <- ip1 I0720 12:28:23.902468 11863 net.cpp:380] ip2 -> ip2 I0720 12:28:23.902484 11863 net.cpp:122] Setting up ip2 I0720 12:28:23.902488 11863 net.cpp:129] Top shape: 100 2 (200) I0720 12:28:23.902492 11863 net.cpp:137] Memory required for data: 67761200 I0720 12:28:23.902496 11863 net.cpp:200] ip2 does not need backward computation. I0720 12:28:23.902504 11863 net.cpp:200] relu1 does not need backward computation. I0720 12:28:23.902508 11863 net.cpp:200] ip1 does not need backward computation. I0720 12:28:23.902510 11863 net.cpp:200] pool2 does not need backward computation. I0720 12:28:23.902514 11863 net.cpp:200] conv2 does not need backward computation. I0720 12:28:23.902520 11863 net.cpp:200] pool1 does not need backward computation. I0720 12:28:23.902523 11863 net.cpp:200] conv1 does not need backward computation. I0720 12:28:23.902526 11863 net.cpp:200] data does not need backward computation. I0720 12:28:23.902529 11863 net.cpp:242] This network produces output ip2 I0720 12:28:23.902534 11863 net.cpp:242] This network produces output label I0720 12:28:23.902545 11863 net.cpp:255] Network initialization done. I0720 12:28:23.910292 11863 net.cpp:744] Ignoring source layer label_data_1_split I0720 12:28:23.911893 11863 net.cpp:744] Ignoring source layer ip2_ip2_0_split I0720 12:28:23.911900 11863 net.cpp:744] Ignoring source layer accuracy I0720 12:28:23.911901 11863 net.cpp:744] Ignoring source layer loss [ INFO] [1532057303.926036328]: Waiting for point cloud to arrive ... [ INFO] [1532057304.195380734]: Received cloud with 307200 points. Processing cloud with: 307200 points. Calculating surface normals ... Using integral images for surface normals estimation ... runtime (normals): 0.0737637 Reversing direction of normals that do not point to at least one camera ... reversed 93044 normals runtime (reverse normals): 0.00334309 After workspace filtering: 307169 points left. After voxelization: 71237 points left. Subsampled 100 at random uniformly. Estimating local reference frames ... Estimated 100 local reference frames in 0.0331791 sec. Finding hand poses ... Found 5 grasp candidate sets in 0.237061 sec. ====> HAND SEARCH TIME: 0.284676 [ INFO] [1532057304.623607567]: Generated 5 grasp candidate sets. Creating grasp images for classifier input ... time for computing 5 point neighborhoods with 0 threads: 0.0406995s [detect_grasps-1] process has died [pid 11863, exit code -11, cmd /home/mastu2/catkin_ws/devel/lib/gpd/detect_grasps __name:=detect_grasps __log:=/home/mastu2/.ros/log/20bf3cba-8bcc-11e8-9b8d-e470b8cdc2dd/detect_grasps-1.log]. log file: /home/mastu2/.ros/log/20bf3cba-8bcc-11e8-9b8d-e470b8cdc2dd/detect_grasps-1*.log all processes on machine have died, roslaunch will exit shutting down processing monitor... ... shutting down processing monitor complete

[atenpas]

Does this happen each time? Do you move the camera or is it fixed?

The error is probably a segmentation fault. This makes it difficult to figure out what's happening.

[ojijinWang]

Thanks. Maybe it is because that i moved the camera. So this code can be only used by a fixed camera, right? I am trying to do the GPD using a in-hand camera. How to fix it or which cpp files are related? And the segmentation may be useless in my project of rotating a value with a background of pipes for WRS, Japan. Can i remove it? or which cpp files are related? Can it be used just like your old "agile_grasp".

By the way, I changed the SVM interface of opencv 3.3 in agile_grasp, so it can be used in ROS Kinetic. https://github.com/ojijinWang/agile_grasp_kinetic.git

apologized for so many questions. Thank you.

[ojijinWang]

solved, the workspace and camera position are very important parameters. It shall be set currently. If wrong, it may lead to empty candidates. And it has died when it generates the candidate images.

I think it is where the problem.

[qyp-robot]

Hello, @ojijinWang ,I have the same error as you, I have reset the network path and hand parameters, but the error still occurs, how do you solve it?

[qyp-robot]

[ojijinWang]

I modified the code. I fixed it by removing the line one by one, and found where caused the crushing. It is difficult the say the detial from your terminal screen. I observe in your terminal you have generated the hand sets, which are candidates, correctly. the problem might be that the hand sets are wrong, or hand sets are empty in sets, or the classifier does not work. you can open the source file, remove the code after generating the hand set, and print out the result of sets to see the problem. by the way, your pc took 24 sec to operate the normals? you need to dual with it.

[qyp-robot]

Thank you for your reply. I have found the problem. It is because VTK is not installed, which causes the visualization to fail. In addition, the code you said runs for 24 seconds. Because of the virtual machine I used, the number of threads allocated was 2 before. Now It has been modified to 8, and the running speed has been improved

[qyp-robot]

But I encountered a new problem. When I simulated in gazebo, the point cloud returned by the camera also included the ground. In the end, the GPD output resulted in a grasp of the ground instead of grasping the object. How is this situation? Solve it? My point cloud input data type is PointCloud2, do you want to filter the point cloud data? Looking forward to your answer, thank you very much