rcl package fails cross compilation

potentialdiffer commented 5 years ago

I am trying to cross compile ROS2 for arm on Debian Stretch. Following the official instructions I am able to start compilation until, however it gets terminated reaching rcl package:

Starting >>> rcl
Starting >>> rosidl_runtime_py
Finished <<< rosidl_runtime_py [1.05s]                                                               
--- stderr: rcl                                                                                      
CMake Warning:
  Manually-specified variables were not used by the project:

    SECURITY

/usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: warning: librosidl_typesupport_fastrtps_c.so, needed by /root/cc_ws/ros2_ws/install/lib/librmw_fastrtps_dynamic_cpp.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/7/../../../../aarch64-linux-gnu/bin/ld: warning: librosidl_typesupport_fastrtps_cpp.so, needed by /root/cc_ws/ros2_ws/install/lib/librmw_fastrtps_dynamic_cpp.so, not found (try using -rpath or -rpath-link)
/root/cc_ws/ros2_ws/install/lib/librmw_fastrtps_dynamic_cpp.so: undefined reference to `rosidl_typesupport_fastrtps_cpp::wstring_to_u16string(std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&, std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> >&)'
/root/cc_ws/ros2_ws/install/lib/librmw_fastrtps_dynamic_cpp.so: undefined reference to `rosidl_typesupport_fastrtps_c::wstring_to_u16string(std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&, rosidl_generator_c__U16String&)'
/root/cc_ws/ros2_ws/install/lib/librmw_fastrtps_dynamic_cpp.so: undefined reference to `rosidl_typesupport_fastrtps_c::u16string_to_wstring(rosidl_generator_c__U16String const&, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >&)'
/root/cc_ws/ros2_ws/install/lib/librmw_fastrtps_dynamic_cpp.so: undefined reference to `rosidl_typesupport_fastrtps_cpp::u16string_to_wstring(std::__cxx11::basic_string<char16_t, std::char_traits<char16_t>, std::allocator<char16_t> > const&, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >&)'
collect2: error: ld returned 1 exit status
make[2]: *** [test/service_fixture__rmw_fastrtps_dynamic_cpp] Error 1
make[1]: *** [test/CMakeFiles/service_fixture__rmw_fastrtps_dynamic_cpp.dir/all] Error 2
make: *** [all] Error 2
---
Failed   <<< rcl    [ Exited with code 2 ]
--- stderr: geometry_msgs                                                          
CMake Warning:
  Manually-specified variables were not used by the project:

    SECURITY

---
Aborted  <<< geometry_msgs

Summary: 133 packages finished [12min 51s]
  1 package failed: rcl
  1 package aborted: geometry_msgs
  113 packages had stderr output: action_msgs actionlib_msgs ament_cmake ament_cmake_auto ament_cmake_clang_format ament_cmake_copyright ament_cmake_core ament_cmake_cppcheck ament_cmake_cpplint ament_cmake_export_definitions ament_cmake_export_dependencies ament_cmake_export_include_directories ament_cmake_export_interfaces ament_cmake_export_libraries ament_cmake_export_link_flags ament_cmake_flake8 ament_cmake_gmock ament_cmake_gtest ament_cmake_include_directories ament_cmake_libraries ament_cmake_lint_cmake ament_cmake_nose ament_cmake_pclint ament_cmake_pep257 ament_cmake_pep8 ament_cmake_pyflakes ament_cmake_pytest ament_cmake_python ament_cmake_ros ament_cmake_target_dependencies ament_cmake_test ament_cmake_uncrustify ament_cmake_xmllint ament_index_cpp ament_lint_auto ament_lint_common builtin_interfaces class_loader composition_interfaces connext_cmake_module console_bridge_vendor example_interfaces fastcdr fastrtps_cmake_module geometry_msgs gmock_vendor gtest_vendor kdl_parser launch_testing_ament_cmake libcurl_vendor libyaml_vendor lifecycle_msgs opensplice_cmake_module orocos_kdl osrf_testing_tools_cpp pendulum_msgs pluginlib poco_vendor python_cmake_module rcl rcl_interfaces rcl_logging_log4cxx rcl_logging_noop rcpputils rcutils resource_retriever rmw rmw_connext_cpp rmw_connext_shared_cpp rmw_implementation rmw_implementation_cmake rmw_opensplice_cpp ros_environment rosgraph_msgs rosidl_adapter rosidl_cmake rosidl_default_generators rosidl_default_runtime rosidl_generator_c rosidl_generator_cpp rosidl_generator_dds_idl rosidl_generator_py rosidl_parser rosidl_typesupport_c rosidl_typesupport_connext_c rosidl_typesupport_connext_cpp rosidl_typesupport_cpp rosidl_typesupport_fastrtps_c rosidl_typesupport_fastrtps_cpp rosidl_typesupport_interface rosidl_typesupport_introspection_c rosidl_typesupport_introspection_cpp rosidl_typesupport_opensplice_c rosidl_typesupport_opensplice_cpp rttest shared_queues_vendor sqlite3_vendor std_msgs std_srvs test_interface_files test_launch_testing test_msgs test_osrf_testing_tools_cpp tinydir_vendor tinyxml2_vendor tinyxml_vendor tlsf uncrustify_vendor unique_identifier_msgs urdf urdfdom urdfdom_headers yaml_cpp_vendor
  95 packages not processed

It seems like librosidl_typesupport_fastrtps_c is not found.

Any idea on this? Or should I just try an Ubuntu VM? Thanks!

potentialdiffer commented 5 years ago

I am getting the same error using a VM with Ubuntu 18.04.

Stapelzeiger commented 5 years ago

Not building the tests should get around this error. Try adding --cmake-args -DBUILD_TESTING=OFF to your colcon build command.

I suspect there is a problem in the rcl cmake scripts. Maybe you can open an inssue on ros2/rcl?

potentialdiffer commented 5 years ago

Thanks! Disabling test building solved the problem for now.

nuclearsandwich commented 5 years ago

@filiperinaldi @lmayencourt do you know which tests are causing these issues so we can enable running tests through cross-compilation?

t0ny-peng commented 4 years ago

@nuclearsandwich Building any test in rcl will result in this error. I'm seeing it too on my computer. I think this ticket deserves reopening.

I did some debug. Here's the whole compiling command of the failed target

/usr/bin/aarch64-linux-gnu-g++ ...

CLICK ME

```shell /usr/bin/aarch64-linux-gnu-g++ \ --sysroot=/home/myname/sysroot_arm64v8_ubuntu_bionic \ CMakeFiles/test_wait__rmw_fastrtps_dynamic_cpp.dir/rcl/test_wait.cpp.o \ -o test_wait__rmw_fastrtps_dynamic_cpp \ -Wl,-rpath,/home/myname/cc_ws/ros2_ws/build/rcl:/home/myname/cc_ws/ros2_ws/install/rmw_fastrtps_dynamic_cpp/lib:/home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_introspection_cpp/lib:/home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_introspection_c/lib:/home/myname/cc_ws/ros2_ws/install/rmw_fastrtps_shared_cpp/lib:/home/myname/cc_ws/ros2_ws/install/rcutils/lib:/home/myname/cc_ws/ros2_ws/install/rosidl_generator_c/lib:/home/myname/cc_ws/ros2_ws/install/rmw/lib:/home/myname/cc_ws/ros2_ws/install/fastrtps/lib:/home/myname/cc_ws/ros2_ws/install/fastcdr/lib:/home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib:/home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_cpp/lib:/home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_c/lib:/home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib:/home/myname/cc_ws/ros2_ws/install/rmw_implementation/lib:/home/myname/cc_ws/ros2_ws/install/rcl_logging_noop/lib \ ../gtest/libgtest_main.a \ ../gtest/libgtest.a \ ../librcl.so \ /home/myname/cc_ws/ros2_ws/install/rmw_fastrtps_dynamic_cpp/lib/librmw_fastrtps_dynamic_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_introspection_cpp/lib/librosidl_typesupport_introspection_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_introspection_c/lib/librosidl_typesupport_introspection_c.so \ /home/myname/cc_ws/ros2_ws/install/rmw_fastrtps_shared_cpp/lib/librmw_fastrtps_shared_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rcutils/lib/librcutils.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_generator_c/lib/librosidl_generator_c.so \ /home/myname/cc_ws/ros2_ws/install/rmw/lib/librmw.so \ /home/myname/cc_ws/ros2_ws/install/fastrtps/lib/libfastrtps.so.1.8.0 \ /home/myname/cc_ws/ros2_ws/install/fastcdr/lib/libfastcdr.so.1.0.9 \ -pthread \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_typesupport_c.so \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_typesupport_cpp.so \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_typesupport_fastrtps_c.so \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_generator_c.so \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_typesupport_fastrtps_cpp.so \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_typesupport_introspection_c.so \ /home/myname/cc_ws/ros2_ws/install/builtin_interfaces/lib/libbuiltin_interfaces__rosidl_typesupport_introspection_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_cpp/lib/librosidl_typesupport_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_c/lib/librosidl_typesupport_c.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_typesupport_c.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_typesupport_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_typesupport_fastrtps_c.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_generator_c.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_typesupport_fastrtps_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_typesupport_introspection_c.so \ /home/myname/cc_ws/ros2_ws/install/rcl_interfaces/lib/librcl_interfaces__rosidl_typesupport_introspection_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rmw_implementation/lib/librmw_implementation.so \ /home/myname/cc_ws/ros2_ws/install/rcl_logging_noop/lib/librcl_logging_noop.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_introspection_cpp/lib/librosidl_typesupport_introspection_cpp.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_introspection_c/lib/librosidl_typesupport_introspection_c.so \ /home/myname/cc_ws/ros2_ws/install/rcutils/lib/librcutils.so \ /home/myname/cc_ws/ros2_ws/install/rosidl_generator_c/lib/librosidl_generator_c.so \ /home/myname/cc_ws/ros2_ws/install/rmw/lib/librmw.so \ -lpthread \ -ldl \ /home/myname/sysroot_arm64v8_ubuntu_bionic/usr/lib/aarch64-linux-gnu/libtinyxml2.so \ /home/myname/sysroot_arm64v8_ubuntu_bionic/usr/lib/aarch64-linux-gnu/libssl.so \ /home/myname/sysroot_arm64v8_ubuntu_bionic/usr/lib/aarch64-linux-gnu/libcrypto.so ```

In which, librmw_fastrtps_dynamic_cpp.so depends on librosidl_typesupport_fastrtps_c.so which is missing. I note that this file isn't linked directly to.

Therefore, I manually included these two missing files in the compile command,

...
/home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_fastrtps_c/lib/librosidl_typesupport_fastrtps_c.so \
/home/myname/cc_ws/ros2_ws/install/rosidl_typesupport_fastrtps_cpp/lib/librosidl_typesupport_fastrtps_cpp.so

... and it compiled, which is expected because all the dependencies are at present.

So the solution could be,

some package should export rosidl_typesupport_fastrtps_c and rosidl_typesupport_fastrtps_cpp as their dependencies using ament_export_dependencie()
or, figure out how the GNU x86_64 compiler can find these two libraries depended by librmw_fastrtps_dynamic_cpp.so even though they are not at present

Output of readelf that might help

CLICK ME

```shell $ aarch64-linux-gnu-readelf ... Dynamic section at Tag Type 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x0000000000000001 (NEEDED) 0x000000000000000e (SONAME) 0x000000000000001d (RUNPATH) ```

further inspection, when doing x86_64 native compilation, the linker log shows that librosidl_typesupport_fastrtps_c.so is eventually found in /home/hao.peng/cc_ws/ros2_ws/install_x86_64/rosidl_typesupport_fastrtps_c/lib/

However, when doing cross compiling, the cross-linker will only search in SYSROOT and will certainly fail because it's not there.

I guess this behavior controlled by these CMake settings in the feneric toolchain file

set(CMAKE_FIND_ROOT_PATH $ENV{ROS2_INSTALL_PATH})
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Here's the log of the succeeded and failed linker log.

linker_output_succeeded_x86_64.txt

linker_output_failed_aarch64.txt

ros-tooling / cross_compile

rcl package fails cross compilation #17