search

BruceCanovas / supersurfel_fusion

Dense RGB-D SLAM system RGB-D SLAM articulated around a supersurfel-based 3D representation for fast, lightweight and compact mapping in indoor environment.
61 stars 11 forks source link

REQUIRED process [supersurfel_fusion_rgbd_benchmark_node-2] has died! #3

Closed hanxiumeng closed 2 years ago

hanxiumeng commented 3 years ago

I've run roslaunch supersurfel_fusion supersurfel_fusion_rgbd_benchmark.launch successfully now！ But when I click the stop check box to cancel the pause, it crashes after only a few seconds, without too many error messages. I don't know what's wrong with that. Here is the full logs:

... logging to /home/robocup3d/.ros/log/ce08ef7c-39ff-11eb-a80e-0492264b1059/roslaunch-robocup3d-22670.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://robocup3d:44397/

SUMMARY

CLEAR PARAMETERS

PARAMETERS

NODES / rqt_reconfigure (rqt_reconfigure/rqt_reconfigure) rviz (rviz/rviz) supersurfel_fusion_rgbd_benchmark_node (supersurfel_fusion/supersurfel_fusion_rgbd_benchmark_node)

auto-starting new master process[master]: started with pid [22680] ROS_MASTER_URI=http://localhost:11311

setting /run_id to ce08ef7c-39ff-11eb-a80e-0492264b1059 process[rosout-1]: started with pid [22691] started core service [/rosout] process[supersurfel_fusion_rgbd_benchmark_node-2]: started with pid [22699] process[rviz-3]: started with pid [22700] process[rqt_reconfigure-4]: started with pid [22701] [ INFO] [1607505640.756203327]: rviz version 1.13.14 [ INFO] [1607505640.756229047]: compiled against Qt version 5.9.5 [ INFO] [1607505640.756234344]: compiled against OGRE version 1.9.0 (Ghadamon) [ INFO] [1607505640.759083239]: Forcing OpenGl version 0. Used GPU 0 0 : compute_capability = 750, cudnn_half = 1, GPU: GeForce RTX 2070 net.optimized_memory = 0 mini_batch = 1, batch = 1, time_steps = 1, train = 0 layer filters size/strd(dil) input output 0 [ INFO] [1607505641.322056506]: Stereo is NOT SUPPORTED [ INFO] [1607505641.322097384]: OpenGl version: 4.6 (GLSL 4.6). [WARN] [1607505641.327481]: Could not find a dynamic reconfigure client named '/supersurfel_fusion_rgbd_benchmark_node' conv 32 3 x 3/ 2 416 x 416 x 3 -> 208 x 208 x 32 0.075 BF 1 conv 64 3 x 3/ 2 208 x 208 x 32 -> 104 x 104 x 64 0.399 BF 2 conv 64 3 x 3/ 1 104 x 104 x 64 -> 104 x 104 x 64 0.797 BF 3 route 2 1/2 -> 104 x 104 x 32 4 conv 32 3 x 3/ 1 104 x 104 x 32 -> 104 x 104 x 32 0.199 BF 5 conv 32 3 x 3/ 1 104 x 104 x 32 -> 104 x 104 x 32 0.199 BF 6 route 5 4 -> 104 x 104 x 64 7 conv 64 1 x 1/ 1 104 x 104 x 64 -> 104 x 104 x 64 0.089 BF 8 route 2 7 -> 104 x 104 x 128 9 max 2x 2/ 2 104 x 104 x 128 -> 52 x 52 x 128 0.001 BF 10 conv 128 3 x 3/ 1 52 x 52 x 128 -> 52 x 52 x 128 0.797 BF 11 route 10 1/2 -> 52 x 52 x 64 12 conv 64 3 x 3/ 1 52 x 52 x 64 -> 52 x 52 x 64 0.199 BF 13 conv 64 3 x 3/ 1 52 x 52 x 64 -> 52 x 52 x 64 0.199 BF 14 route 13 12 -> 52 x 52 x 128 15 conv 128 1 x 1/ 1 52 x 52 x 128 -> 52 x 52 x 128 0.089 BF 16 route 10 15 -> 52 x 52 x 256 17 max 2x 2/ 2 52 x 52 x 256 -> 26 x 26 x 256 0.001 BF 18 conv 256 3 x 3/ 1 26 x 26 x 256 -> 26 x 26 x 256 0.797 BF 19 route 18 1/2 -> 26 x 26 x 128 20 conv 128 3 x 3/ 1 26 x 26 x 128 -> 26 x 26 x 128 0.199 BF 21 conv 128 3 x 3/ 1 26 x 26 x 128 -> 26 x 26 x 128 0.199 BF 22 route 21 20 -> 26 x 26 x 256 23 conv 256 1 x 1/ 1 26 x 26 x 256 -> 26 x 26 x 256 0.089 BF 24 route 18 23 -> 26 x 26 x 512 25 max 2x 2/ 2 26 x 26 x 512 -> 13 x 13 x 512 0.000 BF 26 conv 512 3 x 3/ 1 13 x 13 x 512 -> 13 x 13 x 512 0.797 BF 27 conv 256 1 x 1/ 1 13 x 13 x 512 -> 13 x 13 x 256 0.044 BF 28 conv 512 3 x 3/ 1 13 x 13 x 256 -> 13 x 13 x 512 0.399 BF 29 conv 255 1 x 1/ 1 13 x 13 x 512 -> 13 x 13 x 255 0.044 BF 30 yolo [yolo] params: iou loss: ciou (4), iou_norm: 0.07, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.05 nms_kind: greedynms (1), beta = 0.600000 31 route 27 -> 13 x 13 x 256 32 conv 128 1 x 1/ 1 13 x 13 x 256 -> 13 x 13 x 128 0.011 BF 33 upsample 2x 13 x 13 x 128 -> 26 x 26 x 128 34 route 33 23 -> 26 x 26 x 384 35 conv 256 3 x 3/ 1 26 x 26 x 384 -> 26 x 26 x 256 1.196 BF 36 conv 255 1 x 1/ 1 26 x 26 x 256 -> 26 x 26 x 255 0.088 BF 37 yolo [yolo] params: iou loss: ciou (4), iou_norm: 0.07, obj_norm: 1.00, cls_norm: 1.00, delta_norm: 1.00, scale_x_y: 1.05 nms_kind: greedynms (1), beta = 0.600000 Total BFLOPS 6.910 avg_outputs = 310203 Allocate additional workspace_size = 26.22 MB Loading weights from /home/robocup3d/catkin_ws/src/supersurfel_fusion/config/yolov4-tiny.weights... seen 64, trained: 32012 K-images (500 Kilo-batches_64) Done! Loaded 38 layers from weights-file

try to allocate additional workspace_size = 26.22 MB CUDA allocate done! object names loaded bestKey : 0 Total time = 38 ms nb supersurfels: 1200 stamp: 0 Max model size = 0.119019 MB Mean runtime = 38

Sparse VO success ICP success bestKey : 0 Total time = 34 ms nb supersurfels: 532 stamp: 1 Max model size = 0.119019 MB Mean runtime = 36

Sparse VO success ICP success bestKey : 0 Total time = 25 ms nb supersurfels: 602 stamp: 2 Max model size = 0.119019 MB Mean runtime = 32.3333

Sparse VO success ICP success bestKey : 0 Total time = 24 ms nb supersurfels: 743 stamp: 3 Max model size = 0.119019 MB Mean runtime = 30.25

Sparse VO success ICP success bestKey : 1 Total time = 24 ms nb supersurfels: 921 stamp: 4 Max model size = 0.119019 MB Mean runtime = 29

================================================================================ REQUIRED process [supersurfel_fusion_rgbd_benchmark_node-2] has died! process has died [pid 22699, exit code -11, cmd /home/robocup3d/catkin_ws/devel/lib/supersurfel_fusion/supersurfel_fusion_rgbd_benchmark_node name:=supersurfel_fusion_rgbd_benchmark_node log:=/home/robocup3d/.ros/log/ce08ef7c-39ff-11eb-a80e-0492264b1059/supersurfel_fusion_rgbd_benchmark_node-2.log]. log file: /home/robocup3d/.ros/log/ce08ef7c-39ff-11eb-a80e-0492264b1059/supersurfel_fusion_rgbd_benchmark_node-2*.log Initiating shutdown!

[rqt_reconfigure-4] killing on exit [rviz-3] killing on exit [supersurfel_fusion_rgbd_benchmark_node-2] killing on exit [rosout-1] killing on exit [master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done

BruceCanovas commented 3 years ago

Here are different things you can try to help me solve out the problem:

hanxiumeng commented 3 years ago

@BruceCanovas Hi,I ran successfully with disabling the moving object detection (set enable_mod to false in the launch file),and it seems to be a problem with moving object detection module. After disabling the moving object detection module, the node appears to run stably. Here is the log output,and it looks like the end of the sequence caused the node to stop.

Sparse VO success ICP success bestKey : 369 Total time = 22 ms nb supersurfels: 11231 stamp: 1017 Max model size = 1.1515 MB Mean runtime = 16.8075

CachedAllocator::free_all(): cleaning up after ourselves... ================================================================================REQUIRED process [supersurfel_fusion_rgbd_benchmark_node-2] has died! process has finished cleanly log file: /home/robocup3d/.ros/log/491cd8cc-3d16-11eb-8208-0492264b1059/supersurfel_fusion_rgbd_benchmark_node-2*.log Initiating shutdown!

[rqt_reconfigure-4] killing on exit [rviz-3] killing on exit [supersurfel_fusion_rgbd_benchmark_node-2] killing on exit [rosout-1] killing on exit [master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done

Because of time, I haven't done a single step debugging yet.

BruceCanovas commented 3 years ago

And when using moving object detection without Yolo, does it fail?

hanxiumeng commented 3 years ago

yes,it will fail when using moving object detection without Yolo

hanxiumeng commented 3 years ago

@BruceCanovas I'm sorry to bother you again, I just debug the node in gdb, and here is the error output. Is it a problem about opencv_version or an Opencv program error caused by the odometer not being initialized ? Thank you very much!

[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] (An inappropriate ioctl operation on a device) [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [New Thread 0x7fff5af18700 (LWP 27124)] [New Thread 0x7fff5a717700 (LWP 27125)] [New Thread 0x7fff59f16700 (LWP 27126)] [New Thread 0x7fff59715700 (LWP 27127)] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [New Thread 0x7fff45fff700 (LWP 27134)] [New Thread 0x7fff457fe700 (LWP 27135)] [New Thread 0x7fff44efb700 (LWP 27136)] [tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] [New Thread 0x7fff41dff700 (LWP 27151)] [New Thread 0x7fff419fe700 (LWP 27153)] [New Thread 0x7fff415fd700 (LWP 27152)] [New Thread 0x7fff411fc700 (LWP 27154)] [New Thread 0x7fff409fa700 (LWP 27155)] [New Thread 0x7fff405f9700 (LWP 27156)] [New Thread 0x7fff40dfb700 (LWP 27157)] [New Thread 0x7fff07fff700 (LWP 27158)] [Thread 0x7fff07fff700 (LWP 27158) exited] bestKey : 0 Total time = 40 ms nb supersurfels: 1200 stamp: 0 Max model size = 0.119019 MB Mean runtime = 40

[New Thread 0x7fff07fff700 (LWP 27159)] [Thread 0x7fff07fff700 (LWP 27159) exited] [New Thread 0x7fff07fff700 (LWP 27160)] [Thread 0x7fff07fff700 (LWP 27160) exited] Sparse VO success ICP success bestKey : 0 Total time = 28 ms nb supersurfels: 566 stamp: 1 Max model size = 0.119019 MB Mean runtime = 34

[New Thread 0x7fff07fff700 (LWP 27161)] [Thread 0x7fff07fff700 (LWP 27161) exited] [New Thread 0x7fff07fff700 (LWP 27162)] [Thread 0x7fff07fff700 (LWP 27162) exited]

Thread 1 "supersurfel_fus" received signal SIGSEGV, Segmentation fault. 0x00007fffec1afd8c in cv::optflow::DISOpticalFlowImpl::PatchInverseSearch_ParBody::operator()(cv::Range const&) const () from /usr/local/lib/libopencv_optflow.so.3.4

hanxiumeng commented 3 years ago

look like a problem with DISOpticalFlow

BruceCanovas commented 3 years ago

Ok, could you compile in Debug mode (with the flag -g) and show me the backtrace on gdb please? About OpenCV, do you build with the contrib modules? Because the DIS optical flow is in the contribs. I will try to see if it's related to my code or to OpenCV. In the meantime you can only rely on YOLOv4 to filter out possibly moving object.

hanxiumeng commented 3 years ago

@BruceCanovas hi,sorry to bother you again.Now I find the error code and there are 

    /***** Compute dense optical flow *****/
    dof->calc(gray, gray_estimate, optflow);

of function void MotionDetection::detectMotionSimple in the supersurfel_fusion/core/src/motion_detection.cu

Now it looks like a simple call to one of OpenCV's functions,but I should have been build opencv with the contrib modules successfully.But I will verify if the compilation is successful.

BruceCanovas commented 3 years ago

Maybe this can help: https://github.com/opencv/opencv_contrib/issues/1476

hanxiumeng commented 3 years ago

I have tried all the solutions including the one you recommended. But it do not work out finally, I do have the libopencv_optflow.so.3.4. Maybe here is a problem with my computer, I give it up and I am going to write my own code to detect moving objects without using optical streams. Anyway, thank you so much!

BruceCanovas commented 3 years ago

Ok, I am currently thinking on proposing an alternative more accurate than the DIS optical flow. In the source code, in "core/src/motion_detection.cu", there is a method called detectMotionYoloOnly () that you can use to perform MOD only with YOLO. Some changes may be done to make it more accurate :