simple_demo.launch: "ERROR: Wrong path to settings"

[doctorcolossus]

I am attempting to run simple_demo.launch as described in the README, but it fails with:

[ INFO] [1717669508.068559416]: Elevation mapping node initializing ... 
[ INFO] [1717669509.069437459]: Done.
[ WARN] [1717669509.072226345]: UPDATE GLOBAL MAP
ERROR: Wrong path to settings
[elevation_mapping_0-1] process has died [pid 270, exit code -11, cmd /catkin_ws/devel/lib/elevation_mapping/elevation_mapping /voxel_grid/output:=/robot0/pointcloud /stereo_grey/left/image_raw:=/robot0/image_rect __name:=elevation_mapping_0 __log:=/root/.ros/log/0a2013fa-23ef-11ef-a15e-0242ac110002/elevation_mapping_0-1.log].

1. I don't think I configured any path to any settings, nor should I need to, since the example should work out of the box.
2. The error message should give information about the path (if any) which it received, and why it is "wrong": Does the path not exist? Does the path reference a folder, rather than a file, or vice-versa? Was some other specific path expected? Was the wrong extension given? Was there a problem with the file itself, such as encoding or format? The error message "Wrong path to settings" simply doesn't give any information which a user could use to attempt to troubleshoot or resolve the issue. So this error message needs to be improved.
3. If a path to some settings needs to be configured, then this should be documented in the README, and isn't.

Since the example is meant to serve as documentation of how to get GEM working with my robot, not being able to get it running blocks me from using GEM at all.

I am using the following Dockerfile which I wrote (the base image requires the NVIDIA Container Toolkit and its tag should be adjusted based on your host CUDA version:

FROM nvidia/cuda:12.3.2-cudnn9-devel-ubuntu20.04

ARG DEBIAN_FRONTEND=noninteractive

ARG ROS_DISTRO=noetic

RUN apt-key adv \
      --keyserver "hkp://keyserver.ubuntu.com:80" \
      --recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 && \
    echo "deb http://packages.ros.org/ros/ubuntu focal main" \
      > /etc/apt/sources.list.d/ros-latest.list && \
    apt-get update && \
    apt-get install \
      --yes \
      --no-install-recommends \
      git \
      libeigen3-dev \
      libpcl-dev \
      python3-pip \
      python3.8 \
      ros-$ROS_DISTRO-costmap-2d \
      ros-$ROS_DISTRO-eigen-conversions \
      ros-$ROS_DISTRO-grid-map \
      ros-$ROS_DISTRO-octomap-ros \
      ros-$ROS_DISTRO-pcl-ros \
      ros-$ROS_DISTRO-ros-base \
      ros-$ROS_DISTRO-rqt-gui \
      ros-$ROS_DISTRO-rqt-gui-py \
      ros-$ROS_DISTRO-rviz \
      ros-$ROS_DISTRO-tf-conversions && \
    apt-get clean && \
    mkdir -p /catkin_ws/src && \
    cd /catkin_ws/src && \
    git clone https://github.com/anybotics/kindr && \
    git clone https://github.com/ANYbotics/kindr_ros.git && \
    git clone https://github.com/ZJU-Robotics-Lab/slam_msg.git && \
    git clone https://github.com/ZJU-Robotics-Lab/GEM.git && \
    sed -i "s/++11/++14/g" GEM/layers/CMakeLists.txt && \
    cd /catkin_ws && \
    . /opt/ros/$ROS_DISTRO/setup.sh && \
    catkin_make # https://github.com/ZJU-Robotics-Lab/GEM/issues/2

RUN echo "source /opt/ros/noetic/setup.bash" >> /etc/bash.bashrc && \
    echo "source /catkin_ws/devel/setup.bash" >> /etc/bash.bashrc

... and I made the following shell script to attempt to run the simple_demo.launch, which assumes that:

1. You have named your image 'gem'.
2. test.bag exists in the current working directory.

#!/bin/bash

containerID=$(docker create \
              --env="DISPLAY" \
              --env="QT_X11_NO_MITSHM=1" \
              --tty \
              --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
              --volume="./test.bag:/test.bag" \
              gem)

echo "container ID: $containerID"

hostname=$(docker inspect --format='{{ .Config.Hostname }}' $containerID)

echo "hostname: $hostname"

xhost +local:$hostname

xhost

docker start $containerID

source_ros="source /opt/ros/noetic/setup.bash; "`
          `"source /catkin_ws/devel/setup.bash; "

docker exec --detach $containerID /bin/bash -c \
  "$source_ros"`
  `"sleep 6s; "`
  `"rosbag play test.bag --clock"

docker exec --detach $containerID /bin/bash -c \
  "$source_ros"`
  `"roslaunch /catkin_ws/src/GEM/filter.launch"

sleep 3s

docker exec --interactive --tty $containerID /bin/bash -c \
  "$source_ros"`
  `"roslaunch elevation_mapping_demos simple_demo.launch"

docker stop $containerID

docker rm $containerID

xhost -local:$hostname

The full output of the final docker exec command is:

WARNING: Package name "pointMap_layer" does not follow the naming conventions. It should start with a lower case letter and only contain lower case letters, digits, underscores, and dashes.
started roslaunch server http://970b14a6b0a8:45143/

SUMMARY
========

PARAMETERS
 * /elevation_mapping_0/camera_params_yaml: /home/mav-lab/Pro...
 * /elevation_mapping_0/length_in_x: 12.0
 * /elevation_mapping_0/length_in_y: 12.0
 * /elevation_mapping_0/mahalanobis_distance_threshold: 2.5
 * /elevation_mapping_0/map_frame_id: /robot0/map
 * /elevation_mapping_0/map_saving_file: ./map.pcd
 * /elevation_mapping_0/max_variance: 10000.0
 * /elevation_mapping_0/min_variance: 0.0001
 * /elevation_mapping_0/multi_height_noise: 2e-05
 * /elevation_mapping_0/octomap_obs_resolution: 0.1
 * /elevation_mapping_0/octomap_road_resolution: 0.2
 * /elevation_mapping_0/orthomosaic_saving_dir: ./image/
 * /elevation_mapping_0/position_x: 0.0
 * /elevation_mapping_0/position_y: 0.0
 * /elevation_mapping_0/resolution: 0.1
 * /elevation_mapping_0/robot_base_frame_id: /PandarQT
 * /elevation_mapping_0/robot_id: 0
 * /elevation_mapping_0/robot_local_map_size: 20
 * /elevation_mapping_0/robot_name: robot0
 * /elevation_mapping_0/robot_pose_cache_size: 200
 * /elevation_mapping_0/sensor_frame_id: /PandarQT
 * /elevation_mapping_0/sensor_processor/beam_angle: 0.0006
 * /elevation_mapping_0/sensor_processor/beam_constant: 0.0015
 * /elevation_mapping_0/sensor_processor/ignore_points_above: 0.8
 * /elevation_mapping_0/sensor_processor/ignore_points_below: -5.0
 * /elevation_mapping_0/sensor_processor/min_radius: 0.018
 * /elevation_mapping_0/sensor_processor/type: laser
 * /elevation_mapping_0/submap_saving_dir: ./submaps/
 * /elevation_mapping_0/track_point_frame_id: /PandarQT
 * /elevation_mapping_0/track_point_x: 0.0
 * /elevation_mapping_0/track_point_y: 0.0
 * /elevation_mapping_0/track_point_z: 0.0
 * /elevation_mapping_0/travers_threshold: 0.8
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /
    elevation_mapping_0 (elevation_mapping/elevation_mapping)
    rviz (rviz/rviz)

ROS_MASTER_URI=http://localhost:11311

WARNING: Package name "pointMap_layer" does not follow the naming conventions. It should start with a lower case letter and only contain lower case letters, digits, underscores, and dashes.
process[elevation_mapping_0-1]: started with pid [270]
process[rviz-2]: started with pid [271]
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
[ INFO] [1717669507.870478214]: get robot_id: 0
[ INFO] [1717669507.870892721]: get robot_name: robot0
[ INFO] [1717669507.870904523]: Check Format
[ INFO] [1717669507.870909067]: Check Format Done
[ INFO] [1717669507.874251704]: Elevation mapping node started.
[ INFO] [1717669507.875472576]: Elevation mapping node parameters loading ... 
libGL error: MESA-LOADER: failed to retrieve device information
[intel_init_bufmgr:1027] Error initializing buffer manager.
libGL error: failed to create dri screen
libGL error: failed to load driver: i915
libGL error: failed to open /dev/dri/card0: No such file or directory
libGL error: failed to load driver: iris
libGL error: MESA-LOADER: failed to retrieve device information
[intel_init_bufmgr:1027] Error initializing buffer manager.
libGL error: failed to create dri screen
libGL error: failed to load driver: i915
libGL error: failed to open /dev/dri/card0: No such file or directory
libGL error: failed to load driver: iris
GPU Init mapping:14400
[ INFO] [1717669508.068559416]: Elevation mapping node initializing ... 
[ INFO] [1717669509.069437459]: Done.
[ WARN] [1717669509.072226345]: UPDATE GLOBAL MAP
ERROR: Wrong path to settings
[elevation_mapping_0-1] process has died [pid 270, exit code -11, cmd /catkin_ws/devel/lib/elevation_mapping/elevation_mapping /voxel_grid/output:=/robot0/pointcloud /stereo_grey/left/image_raw:=/robot0/image_rect __name:=elevation_mapping_0 __log:=/root/.ros/log/0a2013fa-23ef-11ef-a15e-0242ac110002/elevation_mapping_0-1.log].

I don't think the libGL errors are related. rviz still works despite them and does show the trajectory, but not the elevation map, of course.

[doctorcolossus]

I was able to resolve the libGL warnings, but the "ERROR: Wrong path to settings" and "process has died" errors persist...

$ ./run/simple_demo.sh
container ID: 505f989e4e004980fa6984985ccbd24ffd349099cd3c9d20aa549967649b8fb0
hostname: 505f989e4e00
non-network local connections being added to access control list
access control enabled, only authorized clients can connect
LOCAL:
SI:localuser:casey
505f989e4e004980fa6984985ccbd24ffd349099cd3c9d20aa549967649b8fb0
WARNING: Package name "pointMap_layer" does not follow the naming conventions. It should start with a lower case letter and only contain lower case letters, digits, underscores, and dashes.
roscore/master is not yet running, will wait for it to start
master has started, initiating launch
... logging to /root/.ros/log/cb79e9ea-450f-11ef-bfba-0242ac110002/roslaunch-505f989e4e00-249.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

WARNING: Package name "pointMap_layer" does not follow the naming conventions. It should start with a lower case letter and only contain lower case letters, digits, underscores, and dashes.
started roslaunch server http://505f989e4e00:45243/

SUMMARY
========

PARAMETERS
 * /elevation_mapping_0/camera_params_yaml: /home/mav-lab/Pro...
 * /elevation_mapping_0/length_in_x: 12.0
 * /elevation_mapping_0/length_in_y: 12.0
 * /elevation_mapping_0/mahalanobis_distance_threshold: 2.5
 * /elevation_mapping_0/map_frame_id: /robot0/map
 * /elevation_mapping_0/map_saving_file: ./map.pcd
 * /elevation_mapping_0/max_variance: 10000.0
 * /elevation_mapping_0/min_variance: 0.0001
 * /elevation_mapping_0/multi_height_noise: 2e-05
 * /elevation_mapping_0/octomap_obs_resolution: 0.1
 * /elevation_mapping_0/octomap_road_resolution: 0.2
 * /elevation_mapping_0/orthomosaic_saving_dir: ./image/
 * /elevation_mapping_0/position_x: 0.0
 * /elevation_mapping_0/position_y: 0.0
 * /elevation_mapping_0/resolution: 0.1
 * /elevation_mapping_0/robot_base_frame_id: /PandarQT
 * /elevation_mapping_0/robot_id: 0
 * /elevation_mapping_0/robot_local_map_size: 20
 * /elevation_mapping_0/robot_name: robot0
 * /elevation_mapping_0/robot_pose_cache_size: 200
 * /elevation_mapping_0/sensor_frame_id: /PandarQT
 * /elevation_mapping_0/sensor_processor/beam_angle: 0.0006
 * /elevation_mapping_0/sensor_processor/beam_constant: 0.0015
 * /elevation_mapping_0/sensor_processor/ignore_points_above: 0.8
 * /elevation_mapping_0/sensor_processor/ignore_points_below: -5.0
 * /elevation_mapping_0/sensor_processor/min_radius: 0.018
 * /elevation_mapping_0/sensor_processor/type: laser
 * /elevation_mapping_0/submap_saving_dir: ./submaps/
 * /elevation_mapping_0/track_point_frame_id: /PandarQT
 * /elevation_mapping_0/track_point_x: 0.0
 * /elevation_mapping_0/track_point_y: 0.0
 * /elevation_mapping_0/track_point_z: 0.0
 * /elevation_mapping_0/travers_threshold: 0.8
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /
    elevation_mapping_0 (elevation_mapping/elevation_mapping)
    rviz (rviz/rviz)

ROS_MASTER_URI=http://localhost:11311

WARNING: Package name "pointMap_layer" does not follow the naming conventions. It should start with a lower case letter and only contain lower case letters, digits, underscores, and dashes.
process[elevation_mapping_0-1]: started with pid [338]
process[rviz-2]: started with pid [340]
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
[ INFO] [1721311961.740269097]: get robot_id: 0
[ INFO] [1721311961.740499773]: get robot_name: robot0
[ INFO] [1721311961.740505434]: Check Format
[ INFO] [1721311961.740509290]: Check Format Done
[ INFO] [1721311961.743713159]: Elevation mapping node started.
[ INFO] [1721311961.744826132]: Elevation mapping node parameters loading ...
GPU Init mapping:14400
[ INFO] [1721311961.950212477]: Elevation mapping node initializing ...
[ INFO] [1721311968.509331829, 1615709258.046477125]: Done.
[ WARN] [1721311968.510689319, 1615709258.046477125]: UPDATE GLOBAL MAP
ERROR: Wrong path to settings
[elevation_mapping_0-1] process has died [pid 338, exit code -11, cmd /catkin_workspace/devel/lib/elevation_mapping/elevation_mapping /voxel_grid/output:=/robot0/pointcloud /stereo_grey/left/image_raw:=/robot0/image_rect __name:=elevation_mapping_0 __log:=/root/.ros/log/cb79e9ea-450f-11ef-bfba-0242ac110002/elevation_mapping_0-1.log].
log file: /root/.ros/log/cb79e9ea-450f-11ef-bfba-0242ac110002/elevation_mapping_0-1*.log

[doctorcolossus]

[chironrong]

Hello, have you solved this problem yet?

[aimiliosnrobotics]

i have exactly the same issue, did you find any solutions ?

[doctorcolossus]

Hey @chironrong and @aimiliosnrobotics. @chironrong, sorry for the slow response - was on vacation.

Unfortunately I never was able to figure out the problem. However, it is nice to know that I am not alone. Perhaps together we will be able to figure something out. This is some motivation for me to try revisiting this, but for now I am busy with other stuff. Please let me know if you guys find any more clues or have any thoughts or theories about this, and of course if @ZJU-Robotics-Lab has any ideas, please chime in.

In the meantime I have had better success with leggedrobotics/elevation_mapping_cupy, so I can recommend that.