Building on Humble

[tmayoff]

I'd like to be able to use OpenDDS with ROS humble, I've put some time to get the library to build on humble (just marking things as unimplemented for now).

I'm getting this error now when trying to use it (launching the topic monitor in rqt), I'm struggling to find some clear documentation to 'creating a rmw' and what needs to be set/enabled or implemented to get something to work.

QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-vscode'
(124728|124795) WARNING: MulticastManager::join: failed to join group 239.255.0.1:7152 on veth08d9df8/0.0.0.0 (0x55a2734e55a8 joined count 4): Unknown error -2

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'Handle's typesupport identifier (rosidl_typesupport_c) is not supported by this library, at ./src/type_support_dispatch.hpp:113'

with this new error message:

  'type support implementation 'rosidl_typesupport_c' does not match 'rosidl_typesupport_opendds_cpp', at /workspaces/ws_humble/rmw_opendds/rmw_opendds_cpp/include/rmw_opendds_cpp/RmwAllocateFree.hpp:84'

rcutils_reset_error() should be called after error handling to avoid this.
<<<

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'type support implementation 'rosidl_typesupport_c' does not match 'rosidl_typesupport_opendds_cpp', at /workspaces/ws_humble/rmw_opendds/rmw_opendds_cpp/include/rmw_opendds_cpp/RmwAllocateFree.hpp:84'

with this new error message:

  'DDSPublisher failed, at /workspaces/ws_humble/rmw_opendds/rmw_opendds_cpp/src/rmw_publisher.cpp:128'

rcutils_reset_error() should be called after error handling to avoid this.
<<<
(124728|124795) WARNING: MulticastManager::join: failed to join group 239.255.0.1:7150 on veth08d9df8/0.0.0.0 (0x55a2734ef448 joined count 4): Unknown error -2
Segmentation fault (core dumped)

[mitza-oci]

Do you have changes in a forked repo? Does something like https://github.com/OpenDDS/rmw_build/blob/master/run_ci_test.sh work on Humble?

[tmayoff]

Woops, forgot to link the fork, https://github.com/tmayoff/rmw_opendds/tree/humble. I didn't use that repo to test, but our repo for work. On humble there were a bunch of build errors, changes to the rmw.h header (and others)

[tmayoff]

There's a lot that needs to be done as prerequisites for getting rmw_build to work in humble

When going through the rmw_build repo I eventually get these build errors, I feel like this might be due to the opendds install I have in the image, being a custom debian repo I'm adding.

root@12e5b606b02d:/opt/workspace# ./rmw_build/build_examples.sh 
not found: "/opt/workspace/install/builtin_interfaces/share/builtin_interfaces/local_setup.bash"
not found: "/opt/workspace/install/unique_identifier_msgs/share/unique_identifier_msgs/local_setup.bash"
not found: "/opt/workspace/install/action_msgs/share/action_msgs/local_setup.bash"
Starting >>> builtin_interfaces
Starting >>> unique_identifier_msgs
--- stderr: builtin_interfaces                                                                                   
In file included from /opt/workspace/build/builtin_interfaces/rosidl_generator_cpp/builtin_interfaces/msg/duration.hpp:7,
                 from /opt/workspace/build/builtin_interfaces/rosidl_typesupport_opendds_cpp/builtin_interfaces/msg/duration__rosidl_typesupport_opendds_cpp.hpp:12,
                 from /opt/workspace/build/builtin_interfaces/rosidl_typesupport_opendds_cpp/builtin_interfaces/msg/dds_opendds/duration__type_support.cpp:8:
/opt/workspace/build/builtin_interfaces/rosidl_generator_cpp/builtin_interfaces/msg/detail/duration__struct.hpp:14:10: fatal error: rosidl_runtime_cpp/bounded_vector.hpp: No such file or directory
   14 | #include "rosidl_runtime_cpp/bounded_vector.hpp"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
gmake[2]: *** [CMakeFiles/builtin_interfaces__rosidl_typesupport_opendds_cpp.dir/build.make:136: CMakeFiles/builtin_interfaces__rosidl_typesupport_opendds_cpp.dir/rosidl_typesupport_opendds_cpp/builtin_interfaces/msg/dds_opendds/duration__type_support.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:307: CMakeFiles/builtin_interfaces__rosidl_typesupport_opendds_cpp.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....
gmake: *** [Makefile:149: all] Error 2
---
Failed   <<< builtin_interfaces [5.47s, exited with code 2]
Aborted  <<< unique_identifier_msgs [7.32s]                               

Summary: 0 packages finished [8.19s]
  1 package failed: builtin_interfaces
  1 package aborted: unique_identifier_msgs
  2 packages had stderr output: builtin_interfaces unique_identifier_msgs
  7 packages not processed

I might try getting the proper docker image/install setup on a ROS2 humble base image first

[tmayoff]

I setup this repo, https://github.com/tmayoff/opendds_ros_infra, can run the enter_workspace.sh script and follow the rest of the rmw_build instructions.

Contains the currently failing workflows to build: https://github.com/tmayoff/opendds_ros_infra/actions/runs/7234496020/job/19710940967

[mitza-oci]

This error

Error: /__w/opendds_ros_infra/opendds_ros_infra/build/action_msgs/rosidl_typesupport_opendds_cpp/action_msgs/msg/dds_opendds/GoalInfo_C.h:8:10: fatal error: builtin_interfaces/msg/dds_opendds/Time_C.h: No such file or directory
    8 | #include "builtin_interfaces/msg/dds_opendds/Time_C.h"

looks like it could be related to code generation when one idl #includes another. I'll work on getting the container running locally so I can try to learn more.

[tmayoff]

The previous error (#include "rosidl_runtime_cpp/bounded_vector.hpp") was solved by adding rosidl_runtime (or something) to the target_link_libraries in one of the files (I forget which now, one of the only changes in the 2 forks I made). But for this one I don't know what to add

[mitza-oci]

I have the file install/builtin_interfaces/include/builtin_interfaces/msg/dds_opendds/Time_C.h so either that's getting built too late or the relevant -I is missing.

[jrw972]

@tmayoff Whats setting are required to make the ROS build infrastructure log the compilation commands?

[tmayoff]

I'm not sure the command itself but I think it's the log_command event handler documented here. I can add that to the CI runs so it's available to see

[tmayoff]

This is the latest run with the debug logging and command outputs: https://github.com/tmayoff/opendds_ros_infra/actions/runs/7564009620/job/20597425937

[mitza-oci]

It shows what colcon is running:

DEBUG:colcon.colcon_core.event_handler.log_command:Invoked command in '/__w/opendds_ros_infra/opendds_ros_infra/build/action_msgs' returned '2': [...snip env...] /usr/bin/cmake --build /__w/opendds_ros_infra/opendds_ros_infra/build/action_msgs -- -j4 -l4

I think this is essentially running make on the results of CMake's "Unix Makefiles" generator. So I think we're looking for a way to see the commands that GNU Make is running.

[tmayoff]

I'm not sure it can be outputted to the screen, this is from the log file.

events.log

[69.646892] (std_msgs) StdoutLine: {'line': b'[ 50%] \x1b[32mBuilding CXX object CMakeFiles/std_msgs__rosidl_typesupport_opendds_cpp.dir/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/header__type_support.cpp.o\x1b[0m\n'}
[69.648123] (std_msgs) StdoutLine: {'line': b'/usr/bin/c++ -DOPENDDS_RAPIDJSON -DOPENDDS_SECURITY -DRCUTILS_ENABLE_FAULT_INJECTION -DROS_PACKAGE_NAME=\\"std_msgs\\" -D__ACE_INLINE__ -Dstd_msgs__rosidl_typesupport_opendds_cpp_EXPORTS -I/opt/workspace/build/std_msgs/rosidl_generator_cpp -I/opt/workspace/build/std_msgs/rosidl_typesupport_opendds_cpp -I/opt/workspace/build/rosidl_typesupport_opendds_cpp -I/opt/workspace/install/rosidl_typesupport_opendds_cpp/include -I/opt/workspace/install/builtin_interfaces/include/builtin_interfaces/msg/dds_opendds -I/opt/workspace/install/builtin_interfaces/include/builtin_interfaces/srv/dds_opendds -I/opt/workspace/install/builtin_interfaces/include/builtin_interfaces/action/dds_opendds -isystem /opt/ros/humble/include/rmw -isystem /opt/ros/humble/include/rosidl_typesupport_interface -isystem /opt/ros/humble/include/rcutils -isystem /opt/ros/humble/include/rosidl_runtime_c -isystem /opt/workspace/install/builtin_interfaces/include/builtin_interfaces -isystem /opt/OpenDDS/tools/rapidjson/include -isystem /usr/local/include/orbsvcs -isystem /opt/ros/humble/include/rosidl_runtime_cpp -isystem /opt/ros/humble/include/fastcdr -isystem /opt/ros/humble/include/rosidl_typesupport_fastrtps_cpp -isystem /opt/ros/humble/include/rosidl_typesupport_fastrtps_c -isystem /opt/ros/humble/include/rosidl_typesupport_introspection_c -isystem /opt/ros/humble/include/rosidl_typesupport_introspection_cpp -g -fPIC -Wall -Wextra -Wpedantic -Wall -Wextra -Wpedantic -std=gnu++14 -MD -MT CMakeFiles/std_msgs__rosidl_typesupport_opendds_cpp.dir/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/header__type_support.cpp.o -MF CMakeFiles/std_msgs__rosidl_typesupport_opendds_cpp.dir/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/header__type_support.cpp.o.d -o CMakeFiles/std_msgs__rosidl_typesupport_opendds_cpp.dir/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/header__type_support.cpp.o -c /opt/workspace/build/std_msgs/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/header__type_support.cpp\n'}
[69.742146] (-) TimerEvent: {}
[69.759407] (std_msgs) StderrLine: {'line': b'In file included from \x1b[01m\x1b[K/opt/workspace/build/std_msgs/rosidl_typesupport_opendds_cpp/std_msgs/msg/header__rosidl_typesupport_opendds_cpp.hpp:23\x1b[m\x1b[K,\n'}
[69.759682] (std_msgs) StderrLine: {'line': b'                 from \x1b[01m\x1b[K/opt/workspace/build/std_msgs/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/header__type_support.cpp:8\x1b[m\x1b[K:\n'}
[69.759831] (std_msgs) StderrLine: {'line': b'\x1b[01m\x1b[K/opt/workspace/build/std_msgs/rosidl_typesupport_opendds_cpp/std_msgs/msg/dds_opendds/Header_C.h:8:10:\x1b[m\x1b[K \x1b[01;31m\x1b[Kfatal error: \x1b[m\x1b[Kbuiltin_interfaces/msg/dds_opendds/Time_C.h: No such file or directory\n'}
[69.759956] (std_msgs) StderrLine: {'line': b'    8 | #include \x1b[01;31m\x1b[K"builtin_interfaces/msg/dds_opendds/Time_C.h"\x1b[m\x1b[K\n'}
[69.760076] (std_msgs) StderrLine: {'line': b'      |          \x1b[01;31m\x1b[K^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\x1b[m\x1b[K\n'}
[69.760204] (std_msgs) StderrLine: {'line': b'compilation terminated.\n'}

[jrw972]

Thanks.

Header_C.h includes builtin_interfaces/msg/dds_opendds/Time_C.h. This means the command should have -I/opt/workspace/install/builtin_interfaces/include but that is not present. So one option is to add this include directory.

However, -I/opt/workspace/install/builtin_interfaces/include/builtin_interfaces/msg/dds_opendds is present so another option is to change the include to just Time_C.h instead of builtin_interfaces/msg/dds_opendds/Time_C.h.

Since Time_C.h is generated, I think we can either 1) look at the way it is generated or 2) add the path to influence the compilation.

I'll keep digging.

[tmayoff]

Some more oddities, generated header for the GoalInfo has this set of includes

include <cstdint>
#include <string>
#include <utility>
#include "builtin_interfaces/msg/dds_opendds/Time_C.h"
#include "tao/Basic_Types.h"
#include "unique_identifier_msgs/msg/dds_opendds/UUID_C.h"
#include "builtin_interfaces/msg/dds_opendds/Time_C.h"
#include "unique_identifier_msgs/msg/dds_opendds/UUID_C.h"

There's duplicated headers,

When I try to add the --filename-only-includes, to opendds_idl to try and accomplish what , it generates this

#include <cstdint>
#include <string>
#include <utility>
#include "builtin_interfaces/msg/dds_opendds/Time_C.h"
#include "tao/Basic_Types.h"
#include "unique_identifier_msgs/msg/dds_opendds/UUID_C.h"
#include "Time_C.h"
#include "UUID_C.h"

[jrw972]

So, did this work, not work, or just weird?

[tmayoff]

More just weird. This was a different error than the Time_C.h ones from before I thought I had gotten further with this, but I tried again and for now fails for the previous reasons Time_C.h related. I tried doing as you suggested changing the include to just Time_C.h but can't as it's including it twice one of which with the full path

[mitza-oci]

These duplicated includes are all in the same file written by opendds_idl? Which one? Maybe there's a bug in --filename-only-includes

[tmayoff]

They are duplicated regardless of the - -filename-only-include. But only some of them change, when that flag is set, there are some other includes coming somewhere else

[tmayoff]

I think I found the header include issue. After that, locally I ran into an include problem with the idl compilation step, but the CI passed, needs further investigation. These are the changes so far to rosidl_typesupport_opendds, https://github.com/OpenDDS/rosidl_typesupport_opendds/pull/50/files

[tmayoff]

Okay so that did fix the build. It now crashes when trying to run anything, I added the run_ci_test.sh to CI, for now I can work through these problems, thanks for the help so far!

[tmayoff]

Okay so I didn't get that much further before getting hit with this series of errors

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'Handle's typesupport identifier (rosidl_typesupport_cpp) is not supported by this library, at ./src/type_support_dispatch.hpp:111'

with this new error message:

  'type support implementation 'rosidl_typesupport_cpp' does not match 'rosidl_typesupport_opendds_cpp', at /home/tyler/src/opendds_ros_infra/src/rmw_opendds/rmw_opendds_cpp/include/rmw_opendds_cpp/RmwAllocateFree.hpp:84'

rcutils_reset_error() should be called after error handling to avoid this.
<<<
(167524|167527) DataWriterImpl::add_association - bit 0 local 010314ac.60d839e1.8e64e713.07000003(90ecdd0a) remote 010314ac.60d839e1.8e64e713.06000004(b6342fcc)

>>> [rcutils|error_handling.c:108] rcutils_set_error_state()
This error state is being overwritten:

  'type support implementation 'rosidl_typesupport_cpp' does not match 'rosidl_typesupport_opendds_cpp', at /home/tyler/src/opendds_ros_infra/src/rmw_opendds/rmw_opendds_cpp/include/rmw_opendds_cpp/RmwAllocateFree.hpp:84'

with this new error message:

  'DDSPublisher failed, at /home/tyler/src/opendds_ros_infra/src/rmw_opendds/rmw_opendds_cpp/src/rmw_publisher.cpp:128'

rcutils_reset_error() should be called after error handling to avoid this.
<<<
terminate called after throwing an instance of 'rclcpp::exceptions::RCLError'
(167524|167530) INFO: MulticastManager::join: joined group 239.255.0.2:7401 on lo/0.0.0.0 (0x7f77e8003c28 joined count 1)
  what():  could not create publisher: DDSPublisher failed, at /home/tyler/src/opendds_ros_infra/src/rmw_opendds/rmw_opendds_cpp/src/rmw_publisher.cpp:128, at /home/tyler/src/opendds_ros_infra/src/rcl/rcl/src/rcl/publisher.c:117
(167524|167530) INFO: MulticastManager::leave: left group 239.255.0.2:7401 on lo/127.0.0.1 (0x7f77e8003c28 joined count 0)
(167524|167527) DataWriterImpl::transport_assoc_done: writer 010314ac.60d839e1.8e64e713.07000003(90ecdd0a) succeeded in associating with reader 010314ac.60d839e1.8e64e713.06000004(b6342fcc)
fish: Job 1, './install/test_pkg/lib/test_pkg…' terminated by signal SIGABRT (Abort)

The error is coming from this line. And this now only happens from the publisher in the test node

EDIT: Okay so I think that was because I was skipping parts of the build to speed up development. It now seems to work, I have to use the non forked examples to get it to work so the run_ci_test.sh won't exit. I'll work on that