search

micro-ROS / micro_ros_setup

Support macros for building micro-ROS-based firmware.
Apache License 2.0
362 stars 132 forks source link

Viability of using RCLCPP as the client library for low resource arm based linux systems #398

Open coalman321 opened 2 years ago

coalman321 commented 2 years ago

Hi, I have a question about the viability of using RCLCPP in place of RCLC for low power / resource constrained arm systems. So far I have a working version of micro_ros for the platform, as well as a working colcon.meta and toolchain, but was curious if it would be possible to use RCLCPP as the client library for the micro_ros client. Given our use case is already C++ on the device, using RCLCPP seems to be a better fit. Especially rather than than having to write a significant wrapper around RCLC. Would this switch even be possible, or are the two client libraries functionally different enough to a point where it would not work? Thank you for your help! (Sorry if this is the wrong place to pose this question)

JanStaschulat commented 2 years ago

Hi @coalman321. Give it a try. The interface for rclcpp is rcl, so there should be no other dependencies (potentially, the type conversion on xrce-dds-client support might be an issue). Mainly you have to watch out for total memory consumption as well as memory fragmentation with RCLCPP. You could make some measurements on a normal linux desktop with ROS 2 Galactic and rclcpp - with your C++ application and check with tools like valgrind, how much (static/dynamic) memory is actually used. This might give you an indication, if 256MB RAM is sufficient.

My question would be - with such a powerfull machine with large memory, why are you not using ROS 2 directly, as you have Linux with Preempt-RT patch already running?

Functionally RCLCPP and RCLC are compliant (RCLC is feature complete: creation of nodes/publisher/subscription/service/client/parameters/...)The RCLC has a (I would argue) a more advanced executor, it offers more deterministic features for real-time behavior. But if you don't need that in your application, start with default RCLCPP Executor. This is the only major difference between RCLCPP and RCLC. Just noting, the rclc-executor pre-allocates memory in the configuration-phase. At runtime there is no dynamic memory allocation. This is not the case for the RCLCPP-Executor (which is using STL-containers).

coalman321 commented 2 years ago

Our biggest issue with RCLC right now is that it doesnt seem to have any real way to deal with C++ and its classing system, something our software is based around. We attempted to build minimal versions of ros2 for the platform, but were unable to get any real traction. Many system dependencies that ROS expects to have access to would not cooperate nor build properly, largely due to the older compiler and kernel being used by the system (something we do not have control over).

That aside, I did make an attempt, by adding RCLCPP into the mcu_ws as well as any other dependencies it has listed and the build does start as expected, but seems to be failing due to missing C++ type support. have not yet had a chance to work out out why, other than the build fails on libstatistics_collector due to missing some of the C++ builtin_interfaces. This is the current error:

fatal error: builtin_interfaces/msg/time.hpp: No such file or directory
 #include "builtin_interfaces/msg/time.hpp"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This seems strange as C++ typesupport is present, and seems to build okay, but isnt producing results. These are the current build args for the generate_lib build.sh script:

colcon build \
        --merge-install \
        --metas $COLCON_META \
        --cmake-args \
        "--no-warn-unused-cli" \
        -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=OFF \
        -DTHIRDPARTY=ON \
        -DBUILD_TESTING=OFF \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_TOOLCHAIN_FILE=$TOOLCHAIN \
        -DCMAKE_VERBOSE_MAKEFILE=ON; \

And this is the list of the current packages in mcu_ws:

colcon list
action_msgs     ros2/rcl_interfaces/action_msgs (ros.ament_cmake)
actionlib_msgs  ros2/common_interfaces/actionlib_msgs   (ros.ament_cmake)
builtin_interfaces      ros2/rcl_interfaces/builtin_interfaces  (ros.ament_cmake)
common_interfaces       ros2/common_interfaces/common_interfaces        (ros.ament_cmake)
composition_interfaces  ros2/rcl_interfaces/composition_interfaces      (ros.ament_cmake)
diagnostic_msgs ros2/common_interfaces/diagnostic_msgs  (ros.ament_cmake)
example_interfaces      ros2/example_interfaces (ros.ament_cmake)
geometry_msgs   ros2/common_interfaces/geometry_msgs    (ros.ament_cmake)
libstatistics_collector libstatistics_collector (ros.ament_cmake)
libyaml_vendor  ros2/libyaml_vendor     (ros.ament_cmake)
lifecycle_msgs  ros2/rcl_interfaces/lifecycle_msgs      (ros.ament_cmake)
micro_ros_msgs  uros/micro_ros_msgs     (ros.ament_cmake)
micro_ros_utilities     uros/micro_ros_utilities        (ros.ament_cmake)
microcdr        eProsima/Micro-CDR      (cmake)
microxrcedds_client     eProsima/Micro-XRCE-DDS-Client  (cmake)
nav_msgs        ros2/common_interfaces/nav_msgs (ros.ament_cmake)
rcl     uros/rcl/rcl    (ros.ament_cmake)
rcl_action      uros/rcl/rcl_action     (ros.ament_cmake)
rcl_interfaces  ros2/rcl_interfaces/rcl_interfaces      (ros.ament_cmake)
rcl_lifecycle   uros/rcl/rcl_lifecycle  (ros.ament_cmake)
rcl_logging_interface   ros2/rcl_logging/rcl_logging_interface  (ros.ament_cmake)
rcl_logging_noop        ros2/rcl_logging/rcl_logging_noop       (ros.ament_cmake)
rcl_yaml_param_parser   ros2/rcl/rcl_yaml_param_parser  (ros.ament_cmake)
rclc    uros/rclc/rclc  (ros.ament_cmake)
rclc_lifecycle  uros/rclc/rclc_lifecycle        (ros.ament_cmake)
rclc_parameter  uros/rclc/rclc_parameter        (ros.ament_cmake)
rclcpp  ros2/rclcpp/rclcpp      (ros.ament_cmake)
rclcpp_components       ros2/rclcpp/rclcpp_components   (ros.ament_cmake)
rclcpp_lifecycle        ros2/rclcpp/rclcpp_lifecycle    (ros.ament_cmake)
rcpputils       ros2/rcpputils  (ros.ament_cmake)
rcutils uros/rcutils    (ros.ament_cmake)
rmw     ros2/rmw/rmw    (ros.ament_cmake)
rmw_implementation      ros2/rmw_implementation/rmw_implementation      (ros.ament_cmake)
rmw_implementation_cmake        ros2/rmw/rmw_implementation_cmake       (ros.ament_cmake)
rmw_microxrcedds        uros/rmw_microxrcedds/rmw_microxrcedds_c        (ros.ament_cmake)
ros2trace       uros/tracetools/ros2trace       (ros.ament_python)
rosgraph_msgs   ros2/rcl_interfaces/rosgraph_msgs       (ros.ament_cmake)
rosidl_adapter  ros2/rosidl/rosidl_adapter      (ros.ament_cmake)
rosidl_cli      ros2/rosidl/rosidl_cli  (ros.ament_python)
rosidl_cmake    ros2/rosidl/rosidl_cmake        (ros.ament_cmake)
rosidl_default_generators       ros2/rosidl_defaults/rosidl_default_generators  (ros.ament_cmake)
rosidl_default_runtime  ros2/rosidl_defaults/rosidl_default_runtime     (ros.ament_cmake)
rosidl_generator_c      ros2/rosidl/rosidl_generator_c  (ros.ament_cmake)
rosidl_generator_cpp    ros2/rosidl/rosidl_generator_cpp        (ros.ament_cmake)
rosidl_generator_dds_idl        ros2/rosidl_dds/rosidl_generator_dds_idl        (ros.ament_cmake)
rosidl_parser   ros2/rosidl/rosidl_parser       (ros.ament_cmake)
rosidl_runtime_c        ros2/rosidl/rosidl_runtime_c    (ros.ament_cmake)
rosidl_runtime_cpp      ros2/rosidl/rosidl_runtime_cpp  (ros.ament_cmake)
rosidl_typesupport_c    uros/rosidl_typesupport/rosidl_typesupport_c    (ros.ament_cmake)
rosidl_typesupport_cpp  uros/rosidl_typesupport/rosidl_typesupport_cpp  (ros.ament_cmake)
rosidl_typesupport_interface    ros2/rosidl/rosidl_typesupport_interface        (ros.ament_cmake)
rosidl_typesupport_introspection_c      ros2/rosidl/rosidl_typesupport_introspection_c  (ros.ament_cmake)
rosidl_typesupport_introspection_cpp    ros2/rosidl/rosidl_typesupport_introspection_cpp        (ros.ament_cmake)
rosidl_typesupport_microxrcedds_c       uros/rosidl_typesupport_microxrcedds/rosidl_typesupport_microxrcedds_c(ros.ament_cmake)
rosidl_typesupport_microxrcedds_c_tests uros/rosidl_typesupport_microxrcedds/test/c     (ros.ament_cmake)
rosidl_typesupport_microxrcedds_cpp     uros/rosidl_typesupport_microxrcedds/rosidl_typesupport_microxrcedds_cpp      (ros.ament_cmake)
rosidl_typesupport_microxrcedds_cpp_tests       uros/rosidl_typesupport_microxrcedds/test/cpp   (ros.ament_cmake)
rosidl_typesupport_microxrcedds_test_msg        uros/rosidl_typesupport_microxrcedds/test/msg   (ros.ament_cmake)
sensor_msgs     ros2/common_interfaces/sensor_msgs      (ros.ament_cmake)
sensor_msgs_py  ros2/common_interfaces/sensor_msgs_py   (ros.ament_python)
shape_msgs      ros2/common_interfaces/shape_msgs       (ros.ament_cmake)
statistics_msgs ros2/rcl_interfaces/statistics_msgs     (ros.ament_cmake)
std_msgs        ros2/common_interfaces/std_msgs (ros.ament_cmake)
std_srvs        ros2/common_interfaces/std_srvs (ros.ament_cmake)
stereo_msgs     ros2/common_interfaces/stereo_msgs      (ros.ament_cmake)
test_interface_files    ros2/test_interface_files       (ros.ament_cmake)
test_msgs       ros2/rcl_interfaces/test_msgs   (ros.ament_cmake)
test_rmw_implementation ros2/rmw_implementation/test_rmw_implementation (ros.ament_cmake)
tracetools      uros/tracetools/tracetools      (ros.ament_cmake)
tracetools_launch       uros/tracetools/tracetools_launch       (ros.ament_python)
tracetools_read uros/tracetools/tracetools_read (ros.ament_python)
tracetools_test uros/tracetools/tracetools_test (ros.ament_cmake)
tracetools_trace        uros/tracetools/tracetools_trace        (ros.ament_python)
trajectory_msgs ros2/common_interfaces/trajectory_msgs  (ros.ament_cmake)
unique_identifier_msgs  ros2/unique_identifier_msgs     (ros.ament_cmake)
visualization_msgs      ros2/common_interfaces/visualization_msgs       (ros.ament_cmake)
coalman321 commented 2 years ago

Okay, so that was a failure on my part. I had forgotten Colcon copies all executables into the install directory the uses the copy when the executable is run. I updated my micro_ros_setup build and it no longer fails at libstatistics_collector. Now its failing at RCLCPP. the current error is as follows:

/home/coalman321/code/micro_ros_ws/firmware/mcu_ws/ros2/rclcpp/rclcpp/src/rclcpp/context.cpp: In member function 'virtual void rclcpp::Context::init(int, const char* const*, const rclcpp::InitOptions&)':
/home/coalman321/code/micro_ros_ws/firmware/mcu_ws/ros2/rclcpp/rclcpp/src/rclcpp/context.cpp:220:24: error: 'using element_type = struct rcl_context_t {aka struct rcl_context_t}' has no member named 'global_arguments'
         &rcl_context_->global_arguments,
                        ^~~~~~~~~~~~~~~~

It looks like the context manager for micro_ros's RCL may not be strictly compatible with the context manager RCLCPP is expecting. The direct cause of this though seems to be a flag not being set for the compilation of RCL. #ifdef RCL_COMMAND_LINE_ENABLED

coalman321 commented 2 years ago

Tracking the resolution of this compilation issue with the RCLCPP group. https://github.com/ros2/rclcpp/issues/1803

JanStaschulat commented 2 years ago

I have also documented first steps for a C++ wrapper around RCLC here: PR https://github.com/ros2/rclc/pull/199 and issue https://github.com/ros2/rclc/issues/126

JanStaschulat commented 2 years ago

Hi @coalman321 cc @pablogs9

Yes, you need to define RCL_COMMAND_LINE_ENABLED https://github.com/micro-ROS/rcl/blob/ae6b2abc43f71ad9e7d1fce6da76d0e2cc88578a/rcl/include/rcl/context.h#L117 Could you check with some print/echo, that the variable is indeed properly configured when compiling micro-ros/rcl?

Tracking the resolution of this compilation issue with the RCLCPP group. ros2/rclcpp#1803

This is an issue with the micro-ros/rcl package not the ros2/rcl package: https://github.com/ros2/rcl/blob/72cee8a9fbda40304ff4f8e7c5d12b6ce6410f8d/rcl/include/rcl/context.h#L117 The file ros2/rcl/include/rcl/context.h does not have the RCL_COMMAND_LINE_ENABLED. but the micro-ros/rcl fork does: https://github.com/micro-ROS/rcl/blob/ae6b2abc43f71ad9e7d1fce6da76d0e2cc88578a/rcl/include/rcl/context.h#L117

JanStaschulat commented 2 years ago

ROS 2 Galactic requires C++17 compiler: https://www.ros.org/reps/rep-2000.html#galactic-geochelone-may-2021-may-2022

Minimum language requirements:
  - C++17
  - Python 3.6

Does your compiler support that?

coalman321 commented 2 years ago

ROS 2 Galactic requires C++17 compiler:

The compiler is an armv7 build of GCC, version 7.3.0, which does support the C++ 17 standard. I am not 100% sure on the python 3.6 support, though I believe the answer is yes. The armv7 version is not currently installed on the build machine though.

Could you check with some print/echo, that the variable is indeed properly configured when compiling micro-ros/rcl?

I am working on checking the cmake variable now I tested, and the variable is being set during build. It was tested via editing the following code in the CMakelists.txt

if(RCL_COMMAND_LINE_ENABLED)
  ament_target_dependencies(${PROJECT_NAME}
    "rcl_yaml_param_parser"
  )
  message("RCL is commandline enabled")
endif()

Colcon does show the message during compile of RCL.

Starting >>> rcl
--- stderr: rcl                                 
CMake Warning at /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcutils/cmake/ament_cmake_export_libraries-extras.cmake:116 (message):
  Package 'rcutils' exports library 'dl' which couldn't be found
Call Stack (most recent call first):
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcutils/cmake/rcutilsConfig.cmake:41 (include)
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rosidl_runtime_c/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rosidl_runtime_c/cmake/rosidl_runtime_cConfig.cmake:41 (include)
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/builtin_interfaces/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/builtin_interfaces/cmake/builtin_interfacesConfig.cmake:41 (include)
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcl_interfaces/cmake/ament_cmake_export_dependencies-extras.cmake:21 (find_package)
  /home/coalman321/code/micro_ros_ws/firmware/mcu_ws/install/share/rcl_interfaces/cmake/rcl_interfacesConfig.cmake:41 (include)
  CMakeLists.txt:10 (find_package)

CMake Warning (dev) at CMakeLists.txt:78 (add_library):
  ADD_LIBRARY called with SHARED option but the target platform does not
  support dynamic linking.  Building a STATIC library instead.  This may lead
  to problems.
This warning is for project developers.  Use -Wno-dev to suppress it.

RCL is commandline enabled
coalman321 commented 2 years ago

ROS 2 Galactic requires C++17 compiler: https://www.ros.org/reps/rep-2000.html#galactic-geochelone-may-2021-may-2022

Minimum language requirements:
  - C++17
  - Python 3.6

Does your compiler support that?

Per this comment, I have some new findings. This weekend, I was able to cross compile up to rclcpp using a stripped down ros installation for the platform. I am still curious about the application for micro_ros, and am willing to re-try the compilation process with both a valid sysroot, and toolchain setup. Although, in this situation, I doubt that it will solve the missing compile flags being an issue, as it was not an issue for normal micro_ros builds.

`