search

ufrj-nautilus / darknet_ros

YOLO ROS: Real-Time Object Detection for ROS
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

darknet_ros is segfaulting with our old pre-trained weights #1

Open matheusrod98 opened 2 years ago

matheusrod98 commented 2 years ago

Bug description

When trying to launch our neural network, it segfaults with exit code -11.

How to reproduce

First, open our simulation with the Lua AUV. roslaunch auv_simulator gazebo.launch namespace:=lua Secondly, launch our nn. roslaunch darknet_ros nautilus_yolov3_front_cam.launch

matheusrod98 commented 2 years ago

Logs

Press Ctrl-C to interrupt
[roslaunch][INFO] 2022-05-09 18:04:20,941: Done checking log file disk usage. Usage is <1GB.
[roslaunch][INFO] 2022-05-09 18:04:20,942: roslaunch starting with args ['/opt/ros/noetic/bin/roslaunch', 'darknet_ros', 'darknet_ros.launch']
[roslaunch][INFO] 2022-05-09 18:04:20,942: roslaunch env is environ({'SHELL': '/bin/bash', 'ROS_VERSION': '1', 'GAZEBO_MASTER_URI': 'http://localhost:11345', 'TMUX': '/tmp/tmux-0/default,51,0', 'PKG_CONFIG_PATH': '/root/catkin_ws/devel/lib/pkgconfig:/opt/ros/noetic/lib/pkgconfig', 'ROS_PYTHON_VERSION': '3', 'HOSTNAME': 'matheus-laptop', 'ROS_PACKAGE_PATH': '/root/catkin_ws/src:/opt/ros/noetic/share', 'ROSLISP_PACKAGE_DIRECTORIES': '/root/catkin_ws/devel/share/common-lisp', 'PWD': '/root/catkin_ws', 'HOME': '/root', 'GAZEBO_PLUGIN_PATH': '/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:', 'ROS_ETC_DIR': '/opt/ros/noetic/etc/ros', 'GAZEBO_MODEL_DATABASE_URI': 'http://models.gazebosim.org', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'CMAKE_PREFIX_PATH': '/root/catkin_ws/devel:/opt/ros/noetic', 'GAZEBO_RESOURCE_PATH': '/usr/share/gazebo-11:/usr/share/gazebo-11:', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'PYTHONPATH': '/root/catkin_ws/devel/lib/python3/dist-packages:/opt/ros/noetic/lib/python3/dist-packages', 'TERM': 'screen', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'TMUX_PANE': '%1', 'DISPLAY': ':0', 'SHLVL': '2', 'ROS_MASTER_URI': 'http://localhost:11311', 'LD_LIBRARY_PATH': '/root/catkin_ws/devel/lib:/opt/ros/noetic/lib:/opt/ros/noetic/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins', 'GAZEBO_MODEL_PATH': '/usr/share/gazebo-11/models:/usr/share/gazebo-11/models:', 'PATH': '/opt/ros/noetic/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'ROS_ROOT': '/opt/ros/noetic/share/ros', 'ROS_DISTRO': 'noetic', 'OGRE_RESOURCE_PATH': '/usr/lib/x86_64-linux-gnu/OGRE-1.9.0', '_': '/opt/ros/noetic/bin/roslaunch', 'LC_CTYPE': 'C.UTF-8', 'ROS_LOG_FILENAME': '/root/.ros/log/1c1c0966-cfdb-11ec-b884-5cc9d38d1361/roslaunch-matheus-laptop-7995.log'})
[roslaunch][INFO] 2022-05-09 18:04:20,943: starting in server mode
[roslaunch.parent][INFO] 2022-05-09 18:04:20,946: starting roslaunch parent run
[roslaunch][INFO] 2022-05-09 18:04:20,947: loading roscore config file /opt/ros/noetic/etc/ros/roscore.xml
[roslaunch][INFO] 2022-05-09 18:04:22,898: Added core node of type [rosout/rosout] in namespace [/]
[roslaunch.config][INFO] 2022-05-09 18:04:22,899: loading config file /root/catkin_ws/src/darknet_ros/darknet_ros/launch/darknet_ros.launch
[roslaunch][INFO] 2022-05-09 18:04:23,601: Added node of type [darknet_ros/darknet_ros] in namespace [/]
[roslaunch][INFO] 2022-05-09 18:04:23,601: ... selected machine [] for node of type [darknet_ros/darknet_ros]
[roslaunch.pmon][INFO] 2022-05-09 18:04:23,667: start_process_monitor: creating ProcessMonitor
[roslaunch.pmon][INFO] 2022-05-09 18:04:23,671: created process monitor <ProcessMonitor(ProcessMonitor-1, initial daemon)>
[roslaunch.pmon][INFO] 2022-05-09 18:04:23,680: start_process_monitor: ProcessMonitor started
[roslaunch.parent][INFO] 2022-05-09 18:04:23,681: starting parent XML-RPC server
[roslaunch.server][INFO] 2022-05-09 18:04:23,684: starting roslaunch XML-RPC server
[roslaunch.server][INFO] 2022-05-09 18:04:23,685: waiting for roslaunch XML-RPC server to initialize
[xmlrpc][INFO] 2022-05-09 18:04:23,698: XML-RPC server binding to 0.0.0.0:0
[xmlrpc][INFO] 2022-05-09 18:04:23,703: Started XML-RPC server [http://matheus-laptop:39865/]
[xmlrpc][INFO] 2022-05-09 18:04:23,703: xml rpc node: starting XML-RPC server
[roslaunch][INFO] 2022-05-09 18:04:23,733: started roslaunch server http://matheus-laptop:39865/
[roslaunch.parent][INFO] 2022-05-09 18:04:23,734: ... parent XML-RPC server started
[roslaunch][INFO] 2022-05-09 18:04:23,744: master.is_running[http://localhost:11311]
[roslaunch][INFO] 2022-05-09 18:04:23,772: master.is_running[http://localhost:11311]
[roslaunch][INFO] 2022-05-09 18:04:23,796: ROS_MASTER_URI=http://localhost:11311
[roslaunch][INFO] 2022-05-09 18:04:23,819: setting /roslaunch/uris/host_matheus_laptop__39865' to http://matheus-laptop:39865/
[roslaunch][INFO] 2022-05-09 18:04:23,838: load_parameters starting ...
[roslaunch][INFO] 2022-05-09 18:04:23,941: ... load_parameters complete
[roslaunch][INFO] 2022-05-09 18:04:23,942: launch_nodes: launching local nodes ...
[roslaunch][INFO] 2022-05-09 18:04:23,949: ... preparing to launch node of type [darknet_ros/darknet_ros]
[roslaunch][INFO] 2022-05-09 18:04:23,949: create_node_process: package[darknet_ros] type[darknet_ros] machine[Machine(name[] env_loader[None] address[localhost] ssh_port[22] user[None] assignable[True] timeout[10.0])] master_uri[http://localhost:11311]
[roslaunch][INFO] 2022-05-09 18:04:23,950: process[darknet_ros-1]: env[{'SHELL': '/bin/bash', 'ROS_VERSION': '1', 'GAZEBO_MASTER_URI': 'http://localhost:11345', 'TMUX': '/tmp/tmux-0/default,51,0', 'PKG_CONFIG_PATH': '/root/catkin_ws/devel/lib/pkgconfig:/opt/ros/noetic/lib/pkgconfig', 'ROS_PYTHON_VERSION': '3', 'HOSTNAME': 'matheus-laptop', 'ROS_PACKAGE_PATH': '/root/catkin_ws/src:/opt/ros/noetic/share', 'ROSLISP_PACKAGE_DIRECTORIES': '/root/catkin_ws/devel/share/common-lisp', 'PWD': '/root/catkin_ws', 'HOME': '/root', 'GAZEBO_PLUGIN_PATH': '/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:', 'ROS_ETC_DIR': '/opt/ros/noetic/etc/ros', 'GAZEBO_MODEL_DATABASE_URI': 'http://models.gazebosim.org', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:', 'CMAKE_PREFIX_PATH': '/root/catkin_ws/devel:/opt/ros/noetic', 'GAZEBO_RESOURCE_PATH': '/usr/share/gazebo-11:/usr/share/gazebo-11:', 'LESSCLOSE': '/usr/bin/lesspipe %s %s', 'PYTHONPATH': '/root/catkin_ws/devel/lib/python3/dist-packages:/opt/ros/noetic/lib/python3/dist-packages', 'TERM': 'screen', 'LESSOPEN': '| /usr/bin/lesspipe %s', 'TMUX_PANE': '%1', 'DISPLAY': ':0', 'SHLVL': '2', 'ROS_MASTER_URI': 'http://localhost:11311', 'LD_LIBRARY_PATH': '/root/catkin_ws/devel/lib:/opt/ros/noetic/lib:/opt/ros/noetic/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins:/usr/lib/x86_64-linux-gnu/gazebo-11/plugins', 'GAZEBO_MODEL_PATH': '/usr/share/gazebo-11/models:/usr/share/gazebo-11/models:', 'PATH': '/opt/ros/noetic/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'ROS_ROOT': '/opt/ros/noetic/share/ros', 'ROS_DISTRO': 'noetic', 'OGRE_RESOURCE_PATH': '/usr/lib/x86_64-linux-gnu/OGRE-1.9.0', '_': '/opt/ros/noetic/bin/roslaunch', 'LC_CTYPE': 'C.UTF-8', 'ROS_LOG_FILENAME': '/root/.ros/log/1c1c0966-cfdb-11ec-b884-5cc9d38d1361/roslaunch-matheus-laptop-7995.log'}]
[roslaunch][INFO] 2022-05-09 18:04:24,351: process[darknet_ros-1]: args[['/root/catkin_ws/devel/lib/darknet_ros/darknet_ros', '__name:=darknet_ros']]
[roslaunch][INFO] 2022-05-09 18:04:24,354: ... created process [darknet_ros-1]
[roslaunch.pmon][INFO] 2022-05-09 18:04:24,355: ProcessMonitor.register[darknet_ros-1]
[roslaunch.pmon][INFO] 2022-05-09 18:04:24,355: ProcessMonitor.register[darknet_ros-1] complete
[roslaunch][INFO] 2022-05-09 18:04:24,356: ... registered process [darknet_ros-1]
[roslaunch][INFO] 2022-05-09 18:04:24,356: process[darknet_ros-1]: starting os process
[roslaunch][INFO] 2022-05-09 18:04:24,356: process[darknet_ros-1]: start w/ args [['/root/catkin_ws/devel/lib/darknet_ros/darknet_ros', '__name:=darknet_ros', '__log:=/root/.ros/log/1c1c0966-cfdb-11ec-b884-5cc9d38d1361/darknet_ros-1.log']]
[roslaunch][INFO] 2022-05-09 18:04:24,356: process[darknet_ros-1]: cwd will be [/root/.ros]
[roslaunch][INFO] 2022-05-09 18:04:24,373: process[darknet_ros-1]: started with pid [8085]
[roslaunch][INFO] 2022-05-09 18:04:24,374: ... successfully launched [darknet_ros-1]
[roslaunch][INFO] 2022-05-09 18:04:24,374: ... launch_nodes complete
[roslaunch.pmon][INFO] 2022-05-09 18:04:24,374: registrations completed <ProcessMonitor(ProcessMonitor-1, started daemon 139661527873280)>
[roslaunch.parent][INFO] 2022-05-09 18:04:24,374: ... roslaunch parent running, waiting for process exit
[roslaunch][INFO] 2022-05-09 18:04:24,374: spin
[roslaunch][ERROR] 2022-05-09 18:04:57,825: [darknet_ros-1] process has died [pid 8085, exit code -11, cmd /root/catkin_ws/devel/lib/darknet_ros/darknet_ros __name:=darknet_ros __log:=/root/.ros/log/1c1c0966-cfdb-11ec-b884-5cc9d38d1361/darknet_ros-1.log].
log file: /root/.ros/log/1c1c0966-cfdb-11ec-b884-5cc9d38d1361/darknet_ros-1*.log
[roslaunch.pmon][INFO] 2022-05-09 18:04:57,837: ProcessMonitor.unregister[darknet_ros-1] starting
[roslaunch.pmon][INFO] 2022-05-09 18:04:57,837: ProcessMonitor.unregister[darknet_ros-1] complete
[roslaunch][INFO] 2022-05-09 18:04:57,838: all processes on machine have died, roslaunch will exit
[roslaunch.pmon][INFO] 2022-05-09 18:04:57,947: ProcessMonitor._post_run <ProcessMonitor(ProcessMonitor-1, started daemon 139661527873280)>
[roslaunch.pmon][INFO] 2022-05-09 18:04:57,947: ProcessMonitor._post_run <ProcessMonitor(ProcessMonitor-1, started daemon 139661527873280)>: remaining procs are []
[roslaunch.pmon][INFO] 2022-05-09 18:04:58,225: ProcessMonitor exit: cleaning up data structures and signals
[roslaunch.pmon][INFO] 2022-05-09 18:04:58,225: ProcessMonitor exit: pmon has shutdown
[roslaunch][INFO] 2022-05-09 18:04:58,263: process monitor is done spinning, initiating full shutdown
[roslaunch][INFO] 2022-05-09 18:04:58,265: runner.stop()
[roslaunch][INFO] 2022-05-09 18:04:58,265: shutting down processing monitor...
[roslaunch][INFO] 2022-05-09 18:04:58,265: shutting down processing monitor <ProcessMonitor(ProcessMonitor-1, stopped daemon 139661527873280)>
[roslaunch.pmon][INFO] 2022-05-09 18:04:58,266: ProcessMonitor.shutdown <ProcessMonitor(ProcessMonitor-1, stopped daemon 139661527873280)>
[roslaunch][INFO] 2022-05-09 18:04:58,266: ... shutting down processing monitor complete
[roslaunch][INFO] 2022-05-09 18:04:58,267: done
[roslaunch.pmon][INFO] 2022-05-09 18:04:58,269: ProcessMonitor.shutdown <ProcessMonitor(ProcessMonitor-1, stopped daemon 139661527873280)>
[rospy.core][INFO] 2022-05-09 18:04:58,272: signal_shutdown [atexit]
matheusrod98 commented 2 years ago

This only happens when using our old pre-trained weights: yolov3-tiny-obj_best.weights, with the standard YOLO weights the neural network is launched as expected. Screenshot from 2022-05-06 23-41-52

matheusrod98 commented 2 years ago

YOLO Model (brhue_yolov3_tiny.yaml)


  config_file:
    name: yolov3-tiny_obj.cfg
  weight_file:
    name: yolov3-tiny-obj_best.weights
  threshold:
    value: 0.3
  detection_classes:
    names: # According to the list of objects from Robosub 2020
      - gman
      - bootlegger
      - path
      - gun
      - badge
      - marker
      - barrel
      - bottle 
      - telephone
      - paper
      - 3d_bottle
      - cash
      - axe
matheusrod98 commented 2 years ago

Launch file:


<launch>

  <!-- Use YOLOv3 -->
  <arg name="network_param_file"         default="$(find darknet_ros)/config/brhue_yolov3_tiny.yaml"/>
  <arg name="image" default="/lua/lua/cameraleft/camera_image" />

  <!-- Include main launch file -->
  <include file="$(find darknet_ros)/launch/nautilus_darknet_ros.launch">
    <arg name="network_param_file"    value="$(arg network_param_file)"/>
    <arg name="image" value="$(arg image)" />
  </include>

</launch>
matheusrod98 commented 2 years ago

@ufrj-nautilus/computer-vision