Conflicting instructions on how to add a custom message

[amilcarlucas]

According to https://micro.ros.org/docs/tutorials/advanced/create_new_type/

I added a new message in firmware/mcu_ws/custom_msg/msg/custom_msg.msg

But when I recompile it I get:

# ros2 run micro_ros_setup create_firmware_ws.sh mbed PORTENTA_H7_M7
Firmware already created. Please delete /microros_ws/firmware folder if you want a fresh installation.

I can not delete the firmware directory!!! It contains the files I was told to create!

[Acuadros95]

To compile, call ros2 run micro_ros_setup build_firmware.sh. The create firmware step only prepares the workspace to build it.

[amilcarlucas]

Thanks, that helped, but now I get:

/microros_ws# ros2 run micro_ros_setup build_firmware.sh
Crosscompiled environment: cleaning path
Building firmware for mbed platform PORTENTA_H7_M7
Traceback (most recent call last):
  File "/usr/local/bin/mbed-tools", line 5, in <module>
    from mbed_tools.cli.main import cli
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/cli/__init__.py", line 7, in <module>
    from mbed_tools.cli.main import cli, LOGGER
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/cli/main.py", line 16, in <module>
    from mbed_tools.cli.configure import configure
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/cli/configure.py", line 10, in <module>
    from mbed_tools.project import MbedProgram
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/__init__.py", line 12, in <module>
    from mbed_tools.project.project import initialise_project, import_project, deploy_project, get_known_libs
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/project.py", line 11, in <module>
    from mbed_tools.project.mbed_program import MbedProgram, parse_url
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/mbed_program.py", line 13, in <module>
    from mbed_tools.project._internal.project_data import (
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/_internal/project_data.py", line 13, in <module>
    from mbed_tools.project._internal.render_templates import (
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/_internal/render_templates.py", line 10, in <module>
    import jinja2
  File "/usr/local/lib/python3.8/dist-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/usr/local/lib/python3.8/dist-packages/jinja2/environment.py", line 15, in <module>
    from jinja2 import nodes
  File "/usr/local/lib/python3.8/dist-packages/jinja2/nodes.py", line 19, in <module>
    from jinja2.utils import Markup
  File "/usr/local/lib/python3.8/dist-packages/jinja2/utils.py", line 642, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)

[amilcarlucas]

@Acuadros95 I fixed it by doing:

rm -Rf firmware
pip3 install markupsafe==2.0.1
ros2 run micro_ros_setup create_firmware_ws.sh mbed PORTENTA_H7_M7

Then I added the custom ROS2 message and did:

ros2 run micro_ros_setup build_firmware.sh

And this time I get:

Finished <<< rcl_interfaces [1min 2s]
Starting >>> composition_interfaces
--- stderr: rmw_implementation
CMake Error at /microros_ws/firmware/micro_ros_mbed/micro_ros_src/install/share/rmw_implementation_cmake/cmake/get_default_rmw_implementation.cmake:60 (message):
  Could not find ROS middleware implementation 'rmw_fastrtps_cpp'.  Choose
  one of the following: rmw_microxrcedds
Call Stack (most recent call first):
  CMakeLists.txt:22 (get_default_rmw_implementation)

---
Failed   <<< rmw_implementation [30.0s, exited with code 1]
Aborted  <<< composition_interfaces [0.81s]
Aborted  <<< std_msgs [1min 3s]
Aborted  <<< rosidl_typesupport_microxrcedds_test_msg [30.1s]
Aborted  <<< test_msgs [25.1s]

Summary: 36 packages finished [3min 9s]
  1 package failed: rmw_implementation
  4 packages aborted: composition_interfaces rosidl_typesupport_microxrcedds_test_msg std_msgs test_msgs
  22 packages had stderr output: action_msgs builtin_interfaces lifecycle_msgs micro_ros_msgs micro_ros_utilities microxrcedds_client rcl_interfaces rcl_logging_interface rcl_logging_noop rcutils rmw rmw_implementation rmw_microxrcedds rosgraph_msgs rosidl_runtime_c rosidl_typesupport_c rosidl_typesupport_microxrcedds_c statistics_msgs std_msgs std_srvs test_msgs unique_identifier_msgs
  20 packages not processed
make: *** [/microros_ws/firmware/micro_ros_mbed/libmicroros.mk:81: /microros_ws/firmware/micro_ros_mbed/micro_ros_src/install] Error 1
ninja: build stopped: subcommand failed.
ERROR: CMake invocation failed!

[Acuadros95]

Yes, just reached the fix. Will set the markupsafe==2.0.1 dependency on the build process.

Your second error is strange, did you call export RMW_IMPLEMENTATION=rmw_fastrtps_cpp in that terminal? That wont work if ros-galactic-fastrtps is not installed.

You should be able to build using export RMW_IMPLEMENTATION=rmw_microxrcedds (May need to delete the firmware folder and call create_firmware again)

[amilcarlucas]

No I did not call export RMW_IMPLEMENTATION=rmw_fastrtps_cpp in terminal. I am using the microros/base:galacticdocker image, and that image did it. If I completely unset all ros environment variables I will not be able to call ros2 run micro_ros_setup build_firmware.sh

What should I unset?

# env | grep ros
AMENT_PREFIX_PATH=/emily_microros/microros_ws/install/micro_ros_setup:/uros_ws/install/micro_ros_setup:/opt/ros/galactic
CMAKE_PREFIX_PATH=/emily_microros/microros_ws/install/micro_ros_setup:/uros_ws/install/micro_ros_setup
COLCON_PREFIX_PATH=/emily_microros/microros_ws/install:/uros_ws/install
PYTHONPATH=/opt/ros/galactic/lib/python3.8/site-packages
LD_LIBRARY_PATH=/opt/ros/galactic/opt/yaml_cpp_vendor/lib:/opt/ros/galactic/lib/x86_64-linux-gnu:/opt/ros/galactic/lib
PATH=/opt/ros/galactic/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
OLDPWD=/emily_microros/microros_ws/firmware/micro_ros_mbed

[Acuadros95]

Just call export RMW_IMPLEMENTATION=rmw_microxrcedds, but this is weird, as rmw_fastrtps_cpp should be available.

[amilcarlucas]

I did this to clean up the environment:

export AMENT_PREFIX_PATH=/emily_microros/microros_ws/install/micro_ros_setup
export CMAKE_PREFIX_PATH=/emily_microros/microros_ws/install/micro_ros_setup
export COLCON_PREFIX_PATH=/emily_microros/microros_ws/install
unset LD_LIBRARY_PATH
export RMW_IMPLEMENTATION=rmw_microxrcedds

and now I get

Building Mbed project...
[20/233] Building CXX object CMakeFiles/micro_ros_mbed.dir/main.cpp.obj
FAILED: CMakeFiles/micro_ros_mbed.dir/main.cpp.obj 
/usr/bin/arm-none-eabi-g++ @CMakeFiles/micro_ros_mbed.dir/main.cpp.obj.rsp -MD -MT CMakeFiles/micro_ros_mbed.dir/main.cpp.obj -MF CMakeFiles/micro_ros_mbed.dir/main.cpp.obj.d -o CMakeFiles/micro_ros_mbed.dir/main.cpp.obj -c ../../../../main.cpp
../../../../main.cpp:3:10: fatal error: rcl/rcl.h: No such file or directory
    3 | #include <rcl/rcl.h>
      |          ^~~~~~~~~~~
compilation terminated.

Why is the include path missing?

[Acuadros95]

Delete the firmware folder, and create/build again.

[amilcarlucas]

I did that, I still get:

Building Mbed project...
[7/291] Performing build step for 'libmicroros_project'
FAILED: libmicroros-prefix/src/libmicroros_project-stamp/libmicroros_project-build ../../../../libmicroros.a 
cd /emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/cmake_build/PORTENTA_H7_M7/develop/GCC_ARM && make -j -f /emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/libmicroros.mk X_CC=/usr/bin/arm-none-eabi-gcc X_AR=/usr/bin/arm-none-eabi-ar X_STRIP=/usr/bin/arm-none-eabi-strip "X_CFLAGS=-Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fmessage-length=0 -fno-exceptions -ffunction-sections -fdata-sections -funsigned-char -fomit-frame-pointer -g3 -mthumb -mfpu=fpv5-d16 -mfloat-abi=softfp -mcpu=cortex-m7" X_CXX=/usr/bin/arm-none-eabi-g++ "X_CXXFLAGS=-Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fmessage-length=0 -fno-exceptions -ffunction-sections -fdata-sections -funsigned-char -fomit-frame-pointer -g3 -mthumb -mfpu=fpv5-d16 -mfloat-abi=softfp -mcpu=cortex-m7" BUILD_DIR=/emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/cmake_build/PORTENTA_H7_M7/develop/GCC_ARM EXTENSIONS_DIR=/emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed && /usr/local/bin/cmake -E touch /emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/cmake_build/PORTENTA_H7_M7/develop/GCC_ARM/libmicroros-prefix/src/libmicroros_project-stamp/libmicroros_project-build
rm -f /emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/mbed_toolchain.cmake; \
cat /emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/mbed_toolchain.cmake.in | \
        sed "s/@CMAKE_C_COMPILER@/\/usr\/bin\/arm-none-eabi-gcc/g" | \
        sed "s/@CMAKE_CXX_COMPILER@/\/usr\/bin\/arm-none-eabi-g++/g" | \
        sed "s/@CFLAGS@/-Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fmessage-length=0 -fno-exceptions -ffunction-sections -fdata-sections -funsigned-char -fomit-frame-pointer -g3 -mthumb -mfpu=fpv5-d16 -mfloat-abi=softfp -mcpu=cortex-m7 -DCLOCK_MONOTONIC=0/g" | \
        sed "s/@CXXFLAGS@/-Wall -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -fmessage-length=0 -fno-exceptions -ffunction-sections -fdata-sections -funsigned-char -fomit-frame-pointer -g3 -mthumb -mfpu=fpv5-d16 -mfloat-abi=softfp -mcpu=cortex-m7 -DCLOCK_MONOTONIC=0/g" | \
        sed "s/@IDF_TARGET@//g" | \
        sed "s/@IDF_PATH@//g" | \
        sed "s/@BUILD_CONFIG_DIR@/\/emily_microros\/microros_ws\/src\/micro_ros_setup\/firmware\/micro_ros_mbed\/cmake_build\/PORTENTA_H7_M7\/develop\/GCC_ARM\/config/g" \
        > /emily_microros/microros_ws/src/micro_ros_setup/firmware/micro_ros_mbed/mbed_toolchain.cmake

[Acuadros95]

Its working on a fresh microros/base:galactic docker, but I think you will be better off using the https://github.com/micro-ROS/micro_ros_mbed repository.

Are you using micro_ros_setup to include custom messages? Let me add this functionality to the micro_ros_mbed repo.

[amilcarlucas]

Yes I satrted out using the https://github.com/micro-ROS/micro_ros_mbed repository and contribuited README fixes to that one.

But yes, I soon realized that I could not add custom messages to it, the documentation required some directories that did not get generated. So I switched to micro_ros_setup. I got it running today with rolling-ros-base-jammy docker, because mbed requires CMake >= 3.19.

It would be great to use the micro_ros_mbed repo to add custom messages, It seams to be smaller than micro_ros_setup.

[Acuadros95]

Added it on both galactic and rolling: https://github.com/micro-ROS/micro_ros_mbed/pull/19 and https://github.com/micro-ROS/micro_ros_mbed/pull/20

Also, you can install the latest cmake version on galactic with the following commands:

apt update
apt install -y git gcc-arm-none-eabi python3 python3-pip sudo git curl apt-transport-https ca-certificates gnupg software-properties-common wget ninja-build
# Get latest CMake
wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | sudo tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null
apt-add-repository 'deb https://apt.kitware.com/ubuntu/ focal main'
apt update
apt install -y cmake

[JanStaschulat]

@amilcarlucas nice to see your interest on micro-ros. In case you need any support, let us know.

[amilcarlucas]

Thanks @JanStaschulat . We (IAV drone team) have been using ROS1 for four years now, but are now moving to ROS2. Nice to see you here too, looking forward to contributing.

[amilcarlucas]

@Acuadros95 I now did add messages and services like described in the page, but after that when I do ros2 run micro_ros_setup build_firmware.sh I get:

Building Mbed project...
ninja: no work to do.

The build script does not look inside the firmware/mcu_ws/* to see if there is work to do. I will now try the new micro_ros_mbed to see if it works there.

[amilcarlucas]

I got it partially running using the micro_ros_mbed repository.

But the second time I issue: mbed-tools compile -m PORTENTA_H7_M7 -t GCC_ARM I always get this error:

FAILED: CMakeFiles/micro_ros_mbed.dir/main.cpp.obj 
/usr/bin/arm-none-eabi-g++ @CMakeFiles/micro_ros_mbed.dir/main.cpp.obj.rsp -MD -MT CMakeFiles/micro_ros_mbed.dir/main.cpp.obj -MF CMakeFiles/micro_ros_mbed.dir/main.cpp.obj.d -o CMakeFiles/micro_ros_mbed.dir/main.cpp.obj -c /emily_microros/uros/micro_ros_mbed/main.cpp
/emily_microros/uros/micro_ros_mbed/main.cpp:3:10: fatal error: rcl/rcl.h: No such file or directory
    3 | #include <rcl/rcl.h>
      |          ^~~~~~~~~~~
compilation terminated.
[39/291] Building CXX object CMakeFiles/micro_ros_mbed.dir/mbed-os/drivers/source/BufferedSerial.cpp.obj

Replacing mbed-tools compile -m PORTENTA_H7_M7 -t GCC_ARM with:

mbed-tools configure -m PORTENTA_H7_M7 -t GCC_ARM
cmake -S . -B cmake_build/PORTENTA_H7_M7/develop/GCC_ARM -GNinja
ninja -C cmake_build/PORTENTA_H7_M7/develop/GCC_ARM

Works the first time issued after a clean. But issuing ninja -C cmake_build/PORTENTA_H7_M7/develop/GCC_ARM afterwords gives the same error as above .

Using cmake --build cmake_build/PORTENTA_H7_M7/develop/GCC_ARM --clean-first also works the first time after a clean and also gives the same error as above when used for the second time.

So far the only solution is to use:

rm -Rf cmake_build include micro_ros_dev micro_ros_src
mbed-tools compile -m PORTENTA_H7_M7 -t GCC_ARM

every single time I compile. There has to be a faster way. Why is cmake and ninja forgetting the include paths?

[Acuadros95]

I could not replicate this on ros:galactic and ros:rolling dockers, mbed-tools compile works multiple times with and without changes on main.cpp. Please don't use microros/base:galactic docker with micro_ros_mbed, as it may interfere with the expected environment variables.

If the problem persist, please elaborate on how to replicate this on a fresh docker environment.

Also, to rebuild the microros lib after including new packages, you can run make -f libmicroros.mk clean before mbed-tools compile

[Acuadros95]

Any updates on this? Can we close?

[amilcarlucas]

This one can be closed. I will open a new one with other issues I'm having

[cse0001]

Thanks, that helped, but now I get:

/microros_ws# ros2 run micro_ros_setup build_firmware.sh
Crosscompiled environment: cleaning path
Building firmware for mbed platform PORTENTA_H7_M7
Traceback (most recent call last):
  File "/usr/local/bin/mbed-tools", line 5, in <module>
    from mbed_tools.cli.main import cli
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/cli/__init__.py", line 7, in <module>
    from mbed_tools.cli.main import cli, LOGGER
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/cli/main.py", line 16, in <module>
    from mbed_tools.cli.configure import configure
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/cli/configure.py", line 10, in <module>
    from mbed_tools.project import MbedProgram
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/__init__.py", line 12, in <module>
    from mbed_tools.project.project import initialise_project, import_project, deploy_project, get_known_libs
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/project.py", line 11, in <module>
    from mbed_tools.project.mbed_program import MbedProgram, parse_url
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/mbed_program.py", line 13, in <module>
    from mbed_tools.project._internal.project_data import (
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/_internal/project_data.py", line 13, in <module>
    from mbed_tools.project._internal.render_templates import (
  File "/usr/local/lib/python3.8/dist-packages/mbed_tools/project/_internal/render_templates.py", line 10, in <module>
    import jinja2
  File "/usr/local/lib/python3.8/dist-packages/jinja2/__init__.py", line 33, in <module>
    from jinja2.environment import Environment, Template
  File "/usr/local/lib/python3.8/dist-packages/jinja2/environment.py", line 15, in <module>
    from jinja2 import nodes
  File "/usr/local/lib/python3.8/dist-packages/jinja2/nodes.py", line 19, in <module>
    from jinja2.utils import Markup
  File "/usr/local/lib/python3.8/dist-packages/jinja2/utils.py", line 642, in <module>
    from markupsafe import Markup, escape, soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsafe/__init__.py)

The root cause with your issue stems from a dependency management conflict between two software ecosystems. According to the error message, your 'markupsafe' was installed using 'pip' at the path /usr/local/lib/python3.8/dist-packages/markupsafe. The 'jinja2' that depends on it was installed using 'apt' at the path /usr/lib/python3/dist-packages/jinja2. The versions of the two are not compatible. In fact, there is a version of 'markupsafe' in the system that 'jinja2' correctly depends on, but the Python interpreter prioritizes the 'markupsafe' installed by 'pip', leading to this issue. There are typically two solutions to this problem: (1) Use 'pip' to uninstall markupsafe, and then reinstall it using 'apt', or uninstall 'jinja2' using 'apt' and reinstall it using 'pip'. (2) Use Python's imp module to customize the path and import the 'markupsafe' from the 'apt' path before importing 'jinja2'. An example is as follows:

import imp
path = ['/usr/lib/python3/dist-packages']
fp, pathname, description = imp.find_module('markupsafe', path)
imp.load_module("markupsafe", fp, pathname, description)

I have developed an automated tool to solve problems like yours and am currently collecting real-world issues. I would really appreciate it if you could reply.