Failed to build on jetson

[ambrosekwok]

Hello,

I tried to build the docker image in jetson nano with is arm64 cpu. Since the base image osrf/ros:humble-desktop is not available for arm64, I changed to ros:humble and removed the ros_tutorials packages. I can build it in my amd64 cpu device but failed in jetson nano. Do you have any ideas to fix it?

Thank you

Here is the modified dockerfile.

FROM ros:humble
#
# How to build this docker image:
#  docker build . -t ros-humble-ros1-bridge-builder
#
# How to build ros-humble-ros1-bridge:
#  # 0.) Start from the ROS 2 Humble system, build a "ros-humble-ros1-bridge/" ROS2 package:
#  docker run --rm ros-humble-ros1-bridge-builder | tar xvzf -
#
# How to use ros-humble-ros1-bridge:
#  # 1.) First start a ROS1 Noetic docker and bring up a GUI terminal, something like:
#  rocker --x11 --user --home --privileged \
#         --volume /dev/shm /dev/shm --network=host -- osrf/ros:noetic-desktop \
#         'bash -c "sudo apt update; sudo apt install -y tilix; tilix"'
#
#  # 2.) Then, start "roscore" inside the ROS1 docker
#  source /opt/ros/noetic/setup.bash
#  roscore
#
#  # 3.) Now, from the ROS2 Humble system, start the ros1 bridge node.
#  source /opt/ros/humble/setup.bash
#  source ros-humble-ros1-bridge/install/local_setup.bash
#  ros2 run ros1_bridge dynamic_bridge
#
#  # 3.) Back to the ROS1 Noetic docker container, run in another terminal tab:
#  source /opt/ros/noetic/setup.bash
#  rosrun rospy_tutorials talker
#
#  # 4.) Finally, from the ROS2 Humble system:
#  source /opt/ros/humble/setup.bash
#  ros2 run demo_nodes_cpp listener
#

# Make sure we are using bash and catching errors
SHELL ["/bin/bash", "-o", "pipefail", "-o", "errexit", "-c"]

# 0.) Bring system up to the latest
RUN apt -y update
RUN apt -y upgrade

# 1.) Temporarily remove ROS2 apt repository
RUN mv /etc/apt/sources.list.d/ros2-latest.list /root/
RUN apt update

# 2.) comment out the catkin conflict
RUN sed  -i -e 's|^Conflicts: catkin|#Conflicts: catkin|' /var/lib/dpkg/status
RUN apt install -f

# 3.) force install these packages
RUN apt download python3-catkin-pkg
RUN apt download python3-rospkg
RUN apt download python3-rosdistro
RUN dpkg --force-overwrite -i python3-catkin-pkg*.deb
RUN dpkg --force-overwrite -i python3-rospkg*.deb
RUN dpkg --force-overwrite -i python3-rosdistro*.deb
RUN apt install -f

# 4.) Install ROS1 stuff
# see https://packages.ubuntu.com/jammy/ros-core-dev
RUN apt -y install ros-core-dev

#
# 4.1) Install additional system packages
# RUN apt -y install g++-12 && \
#     update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-12 12

# 5.) Restore the ROS2 apt repos (optional)
RUN mv /root/ros2-latest.list /etc/apt/sources.list.d/
RUN apt -y update
#RUN apt -y upgrade

# 5.2) Add tf2 support
RUN apt -y install libactionlib-dev
RUN git clone https://github.com/ros/geometry2 && \
    cd geometry2 && \
    git checkout noetic-devel && \
    unset ROS_DISTRO && \
    time colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release
    # colcon test --event-handlers console_direct+ && \
    # colcon test-result

# 5.3) Add tf support
RUN apt -y install libangles-dev
RUN git clone https://github.com/ros-o/geometry.git && \
    source geometry2/install/local_setup.bash && \
    cd geometry && \
    git checkout obese-devel && \
    unset ROS_DISTRO && \
    time colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release && \
    colcon test --event-handlers console_direct+ && \
    colcon test-result

# 6.) Compile ros1_bridge
# ref: https://github.com/ros2/ros1_bridge/issues/391
RUN source geometry2/install/local_setup.bash && \
    source geometry/install/local_setup.bash && \
    source /opt/ros/humble/setup.bash  && \
    mkdir -p /ros-humble-ros1-bridge/src && \
    cd /ros-humble-ros1-bridge/src && \
    git clone https://github.com/ros2/ros1_bridge &&\
    cd ros1_bridge/ && \
    git checkout b9f1739 && \
    cd ../.. && \
    MEMG=$(printf "%.0f" $(free -g | awk '/^Mem:/{print $2}')); \
    NPROC=$(nproc);  MIN=$((MEMG<NPROC ? MEMG : NPROC)); \
    echo "Please wait...  running $MIN concurrent jobs to build ros1_bridge" && \
    time MAKEFLAGS="-j $MIN" colcon build --event-handlers console_direct+ \
      --cmake-args -DCMAKE_BUILD_TYPE=Release
    # colcon test --event-handlers console_direct+ && \
    # colcon test-result

# 7.) Clean up
RUN apt -y clean all; apt -y update

# 8.) Pack all ROS1 dependent libraries
RUN ROS1_LIBS="libxmlrpcpp.so"; \
    ROS1_LIBS="$ROS1_LIBS librostime.so"; \
    ROS1_LIBS="$ROS1_LIBS libroscpp.so"; \
    ROS1_LIBS="$ROS1_LIBS libroscpp_serialization.so"; \
    ROS1_LIBS="$ROS1_LIBS librosconsole.so"; \
    ROS1_LIBS="$ROS1_LIBS librosconsole_log4cxx.so"; \
    ROS1_LIBS="$ROS1_LIBS librosconsole_backend_interface.so"; \
    ROS1_LIBS="$ROS1_LIBS liblog4cxx.so"; \
    ROS1_LIBS="$ROS1_LIBS libcpp_common.so"; \
    ROS1_LIBS="$ROS1_LIBS libb64.so"; \
    ROS1_LIBS="$ROS1_LIBS libaprutil-1.so"; \
    ROS1_LIBS="$ROS1_LIBS libapr-1.so"; \
    cd /ros-humble-ros1-bridge/install/ros1_bridge/lib; \
    for soFile in $ROS1_LIBS; do \
        soFilePath=$(ldd libros1_bridge.so | grep $soFile | awk '{print $3;}'); \
        cp $soFilePath ./; \
    done

# 9.) Spit out ros1_bridge tarball by default when no command is given
RUN tar czf /ros-humble-ros1-bridge.tgz \
    --exclude '*/build/*' --exclude '*/src/*' /ros-humble-ros1-bridge
ENTRYPOINT []
CMD cat /ros-humble-ros1-bridge.tgz

It failed while building the ros1_bridge package

HEAD is now at b9f1739 Fix typo (`services_1_or_2` -> `services_1_to_2`) (#379)
Please wait...  running 6 concurrent jobs to build ros1_bridge
Starting >>> ros1_bridge
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found rmw: 6.1.1 (/opt/ros/humble/share/rmw/cmake)
-- Found Python3: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter 
-- Found rclcpp: 16.0.7 (/opt/ros/humble/share/rclcpp/cmake)
-- Found rosidl_generator_c: 3.1.5 (/opt/ros/humble/share/rosidl_generator_c/cmake)
-- Found rosidl_adapter: 3.1.5 (/opt/ros/humble/share/rosidl_adapter/cmake)
-- Found rosidl_generator_cpp: 3.1.5 (/opt/ros/humble/share/rosidl_generator_cpp/cmake)
-- Using all available rosidl_typesupport_c: rosidl_typesupport_fastrtps_c;rosidl_typesupport_introspection_c
-- Using all available rosidl_typesupport_cpp: rosidl_typesupport_fastrtps_cpp;rosidl_typesupport_introspection_cpp
-- Found rmw_implementation_cmake: 6.1.1 (/opt/ros/humble/share/rmw_implementation_cmake/cmake)
-- Found rmw_fastrtps_cpp: 6.2.5 (/opt/ros/humble/share/rmw_fastrtps_cpp/cmake)
-- Found OpenSSL: /usr/lib/aarch64-linux-gnu/libcrypto.so (found version "3.0.2")  
-- Found FastRTPS: /opt/ros/humble/include  
-- Using RMW implementation 'rmw_fastrtps_cpp' as default
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.2") 
-- Checking for module 'roscpp'
--   Found roscpp, version 1.15.14
-- Checking for module 'std_msgs'
--   Found std_msgs, version 0.5.13
-- Checking for module 'actionlib'
--   Found actionlib, version 1.13.2
-- Checking for module 'actionlib_msgs'
--   Found actionlib_msgs, version 1.13.1
-- Checking for module 'diagnostic_msgs'
--   Found diagnostic_msgs, version 1.13.1
-- Checking for module 'geometry_msgs'
--   Found geometry_msgs, version 1.13.1
-- Checking for module 'nav_msgs'
--   Found nav_msgs, version 1.13.1
-- Checking for module 'rosgraph_msgs'
--   Found rosgraph_msgs, version 1.11.3
-- Checking for module 'sensor_msgs'
--   Found sensor_msgs, version 1.13.1
-- Checking for module 'shape_msgs'
--   Found shape_msgs, version 1.13.1
-- Checking for module 'stereo_msgs'
--   Found stereo_msgs, version 1.13.1
-- Checking for module 'tf'
--   Package 'message_generation', required by 'tf2_msgs', not found
CMake Error at cmake/find_ros1_package.cmake:29 (message):
  find_ros1_package() failed to find 'tf' using pkg_check_modules()
Call Stack (most recent call first):
  CMakeLists.txt:60 (find_ros1_package)

-- Configuring incomplete, errors occurred!
See also "/ros-humble-ros1-bridge/build/ros1_bridge/CMakeFiles/CMakeOutput.log".
--- stderr: ros1_bridge
CMake Error at cmake/find_ros1_package.cmake:29 (message):
  find_ros1_package() failed to find 'tf' using pkg_check_modules()
Call Stack (most recent call first):
  CMakeLists.txt:60 (find_ros1_package)

---
Failed   <<< ros1_bridge [5.15s, exited with code 1]

Summary: 0 packages finished [6.75s]
  1 package failed: ros1_bridge
  1 package had stderr output: ros1_bridge

real    0m7.928s
user    0m5.017s
sys     0m2.876s

Ambrose

[TommyChangUMD]

Hi ambrosekwok,

I have updated Dockerfile to add support for ARM64 cpus. To enable it, replace the first line with:

FROM arm64v8/ros:humble-ros-base-jammy

[ambrosekwok]

Thank you,

I tried the updated Dockerfile with FROM arm64v8/ros:humble-ros-base-jammy, but I still got the same errors while building the ros1_bridge package.

--   Package 'message_generation', required by 'tf2_msgs', not found
CMake Error at cmake/find_ros1_package.cmake:29 (message):
  find_ros1_package() failed to find 'tf' using pkg_check_modules()
Call Stack (most recent call first):
  CMakeLists.txt:60 (find_ros1_package)

-- Configuring incomplete, errors occurred!
See also "/ros-humble-ros1-bridge/build/ros1_bridge/CMakeFiles/CMakeOutput.log".
--- stderr: ros1_bridge
CMake Error at cmake/find_ros1_package.cmake:29 (message):
  find_ros1_package() failed to find 'tf' using pkg_check_modules()
Call Stack (most recent call first):
  CMakeLists.txt:60 (find_ros1_package)

---
Failed   <<< ros1_bridge [4.35s, exited with code 1]

[TommyChangUMD]

That's strange. Are you using the latest Dockerfile from the github directly and modifying only the first line? Don't modify anything else.

[ambrosekwok]

Yes, I use the latest Dockerfile provided and only modified the first line for arm64 base image.

And I tried in jetson orin nx and m2 mac mini, both failed to build the image.

[TommyChangUMD]

OK. In that case, can you post the complete docker build output? You may want to remove the cached result first. Something like:

   docker rmi ros-humble-ros1-bridge-builder
   docker  image prune

[ambrosekwok]

Thanks,

Here is the docker build output.

build.log

[TommyChangUMD]

Thanks a lot! I have updated Dockerfile. It should work this time. You can use the Dockerfile as-is. There is no need to replace the first line as the base image is multi-platform already.

[ambrosekwok]

Unfortunately, this still not work on my jetson and m2 mac mini.

build.log

[TommyChangUMD]

OK. I was hoping that using the binary packages would solve the issue. At this point, we would need to dig deeper. Can you try the following:

0.) Temporarily comment out steps 8 and 9 in the Dockerfile so that the docker image builds.

1.) Log into the created docker container $ docker run -it --rm ros-humble-ros1-bridge-builder bash

While inside the container, do the following: 2.) Check to see if tf2_msgs-related packages are installed

$ dpkg -l | grep libtf2
libtf2-2d:amd64
libtf2-bullet-dev:amd64
libtf2-dev:amd64
libtf2-eigen-dev:amd64
libtf2-geometry-msgs-dev:amd64
libtf2-kdl-dev:amd64
libtf2-msgs-dev:amd64
libtf2-ros-dev:amd64
libtf2-ros1d:amd64
libtf2-sensor-msgs-dev:amd64

3.) Check to see if the the ft2-msgs cmake files are installed

$ ls /usr/share/tf2_msgs/
action  cmake  msg  package.xml  srv

$ ls /usr/share/tf2_msgs/cmake/tf2_msgsConfig.cmake 
/usr/share/tf2_msgs/cmake/tf2_msgsConfig.cmake

4.) Check to see if the the ft2-msgs configure setting can be found

$ pkg-config --list-all | grep tf2
tf2                       tf2 - Description of tf2
tf2_bullet                tf2_bullet - Description of tf2_bullet
tf2_eigen                 tf2_eigen - Description of tf2_eigen
tf2_geometry_msgs         tf2_geometry_msgs - Description of tf2_geometry_msgs
tf2_kdl                   tf2_kdl - Description of tf2_kdl
tf2_msgs                  tf2_msgs - Description of tf2_msgs
tf2_py                    tf2_py - Description of tf2_py
tf2_ros                   tf2_ros - Description of tf2_ros
tf2_sensor_msgs           tf2_sensor_msgs - Description of tf2_sensor_msgs
tf2_tools                 tf2_tools - Description of tf2_too

4.) Check the ft2-msgs configure setting

$ pkg-config --modversion tf2_msgs
0.7.5

[TommyChangUMD]

From the log, it says: Package 'message_generation', required by 'tf2_msgs', not found

So, it looks like tf2_msgs may not be the issue. The issue is that message_generation is missing. Make sure ros-message-generation is installed. I've updated the Dockerfile, please try again let me know if it works.

[ambrosekwok]

Checked the ros-message-generation was installed with ros-core-dev or ros-robot-dev. But pkg-config --list-all | grep message_generation show nothing.

And I found that some pkg-config files seems located in wrong directory on the jetson.

$ ls /usr/lib/x86_64-linux-gnu/pkgconfig/
eigen_stl_containers.pc  gencpp.pc  genlisp.pc  message_generation.pc  roslisp.pc

I copied the files to /usr/lib/aarch64-linux-gnu/pkgconfig/ and pkg-config can find it now.

$ cp /usr/lib/x86_64-linux-gnu/pkgconfig/* /usr/lib/aarch64-linux-gnu/pkgconfig/
$ pkg-config --list-all | grep message_generation
message_generation        message_generation - Description of message_generation

Able to build ros1_bridge with tf message types after that.

$ ros2 run ros1_bridge dynamic_bridge --print-pairs | grep tf
  - 'tf2_msgs/msg/TF2Error' (ROS 2) <=> 'tf2_msgs/TF2Error' (ROS 1)
  - 'tf2_msgs/msg/TFMessage' (ROS 2) <=> 'tf2_msgs/TFMessage' (ROS 1)
  - 'tf2_msgs/msg/TFMessage' (ROS 2) <=> 'tf/tfMessage' (ROS 1)
  - 'tf2_msgs/srv/FrameGraph' (ROS 2) <=> 'tf2_msgs/FrameGraph' (ROS 1)

[TommyChangUMD]

I've updated Dockerfile to include the fix you found. Could you test it? Thanks.

[ambrosekwok]

The updated Dockerfile work, thank you. Also checked I can receive topic with ros2 topic echo /tf.

Thanks a lot.