building PX4 with colcon on ubuntu22 - ros2 humble: ROS2 interface proto can't find px4 module

beniaminopozzan commented 1 year ago

Describe the bug

Starting from #20689 it should be possible to built px4 msgs directly from the main repo using colcon without need of the package px4_msgs. The package is indeed built, all PX4 msgs seems to be available, they can be listed by px4 interface package px4 and the message definition can be inspected by px4 interface show .... However, px4 interface proto fails and returns ModuleNotFoundError as in https://stackoverflow.com/questions/74859379/ros2-throws-modulenotfounderror-when-calling-interface-proto-for-custom-msg I got this error only on an Ubuntu 22.04 machine running ros humble. On another machine with Ubuntu 20.04 with ros foxy everything works smoothly.
Secondary problem: the package name is px4 while the usual px4 messages are under the package px4_msgs. Assuming the the compilation works, adapting https://github.com/PX4/PX4-Autopilot/blob/main/src/modules/microdds_client/dds_topics.yaml is required.

To Reproduce

Steps to reproduce the behavior, assuming: ros2 humble installed from binaries, PX4 toolchain configured.

Create a new colcon workspace

mkdir -p ~/px4_colcon_ws/src
cd ~/px4_colcon_ws/src

Clone PX4-Autopilot

git clone https://github.com/PX4/PX4-Autopilot.git --recursive

Modify at least one entry of https://github.com/PX4/PX4-Autopilot/blob/main/src/modules/microdds_client/dds_topics.yaml such that the required message type belongs to px4 package instead of px4_msgs, for example VehicleOdometry

diff --git a/src/modules/microdds_client/dds_topics.yaml b/src/modules/microdds_client/dds_topics.yaml
index 3a871848bb..90506a5821 100644
--- a/src/modules/microdds_client/dds_topics.yaml
+++ b/src/modules/microdds_client/dds_topics.yaml
@@ -36,7 +36,7 @@ publications:
    type: px4_msgs::msg::VehicleLocalPosition

  - topic: /fmu/out/vehicle_odometry
-    type: px4_msgs::msg::VehicleOdometry
+    type: px4::msg::VehicleOdometry

  - topic: /fmu/out/vehicle_status
    type: px4_msgs::msg::VehicleStatus

the diff file adapt-dds_topic.patch.txt can be applied with git apply << adapt-dds_topic.patch.txt

source the ros2 environment
```
source /opt/ros/humble/setup.bash
```
compile the workspace
```
colcon build --symlink-install
```
source the workspace
```
source install/local_setup.bash
```

It is now possible to list all message associated to the newly compiled px4 package

ros2 interface package px4

which will return something like

px4/msg/InputRc
px4/msg/HeaterStatus
px4/msg/QshellRetval
px4/msg/VehicleAngularVelocity
px4/msg/Airspeed
px4/msg/SensorGps
.
.
.

And it is possible to inspect the message definitions

ros2 interface show px4/msg/Ping

which will return

uint64 timestamp            # time since system start (microseconds)
uint64 ping_time            # Timestamp of the ping packet
uint32 ping_sequence        # Sequence number of the ping packet
uint32 dropped_packets      # Number of dropped ping packets
float32 rtt_ms              # Round trip time (in ms)
uint8 system_id             # System ID of the remote system
uint8 component_id          # Component ID of the remote system

However, when trying to create a prototype of any message

ros2 interface proto px4/msg/Ping

you will get the error

Traceback (most recent call last):
  File "/opt/ros/humble/bin/ros2", line 33, in <module>
    sys.exit(load_entry_point('ros2cli==0.18.5', 'console_scripts', 'ros2')())
  File "/opt/ros/humble/lib/python3.10/site-packages/ros2cli/cli.py", line 89, in main
    rc = extension.main(parser=parser, args=args)
  File "/opt/ros/humble/lib/python3.10/site-packages/ros2interface/command/interface.py", line 35, in main
    return extension.main(args=args)
  File "/opt/ros/humble/lib/python3.10/site-packages/ros2interface/verb/proto.py", line 33, in main
    yaml = interface_to_yaml(args.type)
  File "/opt/ros/humble/lib/python3.10/site-packages/ros2interface/api/__init__.py", line 46, in interface_to_yaml
    interface = utilities.get_interface(identifier)
  File "/opt/ros/humble/lib/python3.10/site-packages/rosidl_runtime_py/utilities.py", line 23, in get_interface
    return import_message_from_namespaced_type(get_namespaced_type(identifier))
  File "/opt/ros/humble/lib/python3.10/site-packages/rosidl_runtime_py/import_message.py", line 30, in import_message_from_namespaced_type
    module = importlib.import_module(
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'px4'

At the same time, if a simulation is started make px4_sitl gazebo, the microDDS agent enabled MicroXRCEAgent udp4 -p 8888, then all topics are available but they cannot be echoed.
```
ros2 topic echo /fmu/out/vehicle_odometry
```
will return the error
```
The message type 'px4/msg/VehicleOdometry' is invalid
```

Expected behavior

The expected behavior can be easily seen using px4_msgs packages are message container and repeating all the above test with it. In particular,

ros2 interface proto px4_msgs/msg/TaskStackInfo

gives

"timestamp: 0
ping_time: 0
ping_sequence: 0
dropped_packets: 0
rtt_ms: 0.0
system_id: 0
component_id: 0
"

while echoing a topic gives the expected echo.

DronecodeBot commented 1 year ago

This issue has been mentioned on Discussion Forum for PX4, Pixhawk, QGroundControl, MAVSDK, MAVLink. There might be relevant details there:

https://discuss.px4.io/t/problem-compiling-px4-v13-3-and-14-beta1-and-beta2-with-ros2-humble-and-ubuntu-22/32315/1

beniaminopozzan commented 1 year ago

I was able to track down the issue to rosidl_generate_interfaces. If https://github.com/PX4/PX4-Autopilot/blob/fc357921fd69e574a3f596be25faa8f4d9cdcea9/platforms/ros2/CMakeLists.txt#L63-L65 is called from src/PX4-Autopilot/platforms/ros2/CMakeLists.txt, which in return is called by https://github.com/PX4/PX4-Autopilot/blob/fc357921fd69e574a3f596be25faa8f4d9cdcea9/CMakeLists.txt#L447-L448 then the issue occurs. As soon as the content of src/PX4-Autopilot/platforms/ros2/CMakeLists.txt is copied in the main CMakeLists.txt file in the root of PX4, then everything works.

I confirmed once again that the issue occurs on Ubuntu 22, ROS Humble while on Ubuntu 20, ROS Foxy it works.

At this point I don't know if this is a ROS issue, CMAKE issue, colcon issue.

My guess is that project definition, and message generation should be performed in the same CMakeLists file. This kind of confirms the guess: https://answers.ros.org/question/361289/ros2-components-registration-from-subdirectory-cmakelists-file/

This is probably because in CMake each call to add_subdirectory() creates a new scope, meaning CMake variables set in the subdirectory are not visible to the parent directory.

DaesungAn commented 1 year ago

I resolved same issue. After applying https://github.com/ament/ament_cmake/pull/431, re-build all and it worked.

beniaminopozzan commented 1 year ago

Hi @DaesungAn ! You are right, I installed ros rolling (base version) and made a clean build. It build successfully and ros2 interface proto px4_msgs/msg/TaskStackInfo gives the proper output.

At that point I tried to replace https://github.com/PX4/px4_msgs with this in tree package. I did the following test:

ROS distro: rolling
Workspace A: just https://github.com/PX4/px4_msgs
Workspace B: just PX4-Autopilot _with the package name changed to px4_msgs_
Workspace C: just https://github.com/PX4/px4_ros_com

I independently compiled workspace A and B, both of them extending only the bare ros2 rolling. The compilations were successful and I could get ros2 interface proto of the px4 messages from both workspaces. Then the real test:

Compiling workspace C with Workspace A sourced (namely, using the standard https://github.com/PX4/px4_msgs) works just fine (as expected).
Compiling workspace C with Workspace B sourced (namely, using the in tree PX4 package) gives me
```
fatal error: px4_msgs/msg/offboard_control_mode.hpp: No such file or directory
```
on all px4_msgs that I try to include and use.

@DaesungAn Did you try something similar?

At this point I went and see the content of the install directories of Workspace A and B, they should be the same, but they are not, in particular workspace B is missing files that I presume are related to the command ament_export_dependencies(rosidl_default_runtime). However, even adding it just before ament_package() in https://github.com/PX4/PX4-Autopilot/blob/main/platforms/ros2/cmake/finalize.cmake does not change the result. I still think that part of the output of rosidl_generate_interfaces is hidden from ament_cmake due to the use of add_subdirectory On the left workspace B, on the right workspace A

GenCyber-Teacher commented 1 year ago

"As soon as the content of src/PX4-Autopilot/platforms/ros2/CMakeLists.txt is copied in the main CMakeLists.txt file in the root of PX4, then everything works." @beniaminopozzan: would you please show me which lines I need to copy to solve the problem? Thanks

aa-calvo commented 1 year ago

I was getting the same exception from the terminal in which I was trying to call ros2 topic echo ... because I wasn't sourcing the install/setup.sh of the workspace in which I was building px4_msgs. I use a single workspace, so this probably doesn't apply to everyone, but hopefully it helps out a few.

Cristian-wp commented 1 year ago

Hi @beniaminopozzan , I am trying to repeat the change you made to avoid the add_subdirectory(platforms/${PX4_PLATFORM}) problem in PX4-Autopilot/CMakeLists.txt. But I can not understand what did you do...If I simply copy&paste the from PX4-Autopilot/platform/ros2/CMakeLists.txt I get an error about the px4 target already exists. I have trying to resolve it, but I can not understand what I have to do...can you please help?

PX4 / PX4-Autopilot