Humble CI is broken

[rhaschke]

Humble CI is failing with the following error:

--- stderr: moveit_ros_planning_interface
/home/runner/work/moveit2/moveit2/.work/target_ws/src/moveit2/moveit_ros/planning_interface/move_group_interface/src/move_group_interface.cpp:54:10: fatal error: moveit_msgs/srv/execute_known_trajectory.hpp: No such file or directory
   54 | #include <moveit_msgs/srv/execute_known_trajectory.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

I don't understand why. The linked GHA build succeeded on Dec 20, but fails now (I just retriggered the build). Hence, it is not related to the source itself. It must be related to GHA, industrial_ci, or the docker image. However, a local build using industrial_ci and the humble docker image succeeded for me. I' don't know what I'm missing...

[tylerjw]

This broke a while ago, I think @AndyZe is working on it.

[rhaschke]

It was still working on Dec 20, which I don't consider a while ago. First failure in GHA was on Jan 4. Are you sure, somebody is working on it already?

[AndyZe]

I looked at it briefly but really have no idea. Seems like a cache needs to be cleaned out, maybe.

[rhaschke]

A cache wasn't used - according to the logs.

[vatanaksoytezer]

Seems https://github.com/ros-planning/moveit_msgs/pull/149 broke it.

[rhaschke]

Thanks! This explains it. Obviously, @AndyZe forgot to remove the include. Open question: why is moveit_msgs installed as a system dependency in rolling docker images and thus hides this bug?

[vatanaksoytezer]

Usually, we try to avoid building from source, and thus use binaries if available and not updated regularly. Source build of moveit_msgs has been deleted from rolling but probably forgotten to be backported to humble, thus hides the bug.

Having said that I think we should strive for good caching (ie never build upstream workspaces if there are no changes) and still use source builds as the main development path. I don't mind adding a separate binary build as well, Maybe we can run full binary builds regularly instead of every PR.

[rhaschke]

Source build of moveit_msgs has been deleted from rolling.

Looks like this should be reverted. Usually, we want to test against latest moveit_msgs, don't we? If these have changes they should get released in sync with MoveIt.

[henningkayser]

I think we only want to test against the latest moveit_msgs if there are staging changes and we probable want to keep them enabled in Rolling. For the stable humble release, this usually is not the case. If we want to support source builds we either have to backport the breaking changes in MoveIt (not great, imo), or we maintain a separate humble branch in moveit_msgs.

[AndyZe]

Actually Humble is passing again at the moment :+1:

[moriarty]

:thinking: this is back?

https://build.ros2.org/job/Hbin_uJ64__moveit_ros_planning_interface__ubuntu_jammy_amd64__binary/51/console

23:23:37 /tmp/binarydeb/ros-humble-moveit-ros-planning-interface-2.5.4/move_group_interface/src/move_group_interface.cpp:54:10: fatal error: moveit_msgs/srv/execute_known_trajectory.hpp: No such file or directory
23:23:41    54 | #include <moveit_msgs/srv/execute_known_trajectory.hpp>
23:23:41       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
23:23:41 compilation terminated.

[moriarty]

Why is the build farm using version 2.5.4 :thinking:

23:29:21 # BEGIN SUBSECTION: get sourcedeb
23:29:23 Invoking '/usr/bin/python3 /tmp/ros_buildfarm/ros_buildfarm/wrapper/apt.py source --download-only --only-source ros-humble-moveit-ros-planning-interface=2.5.4-1jammy'
23:29:24 Invoking 'apt-get source --download-only --only-source ros-humble-moveit-ros-planning-interface=2.5.4-1jammy'

[moriarty]

https://raw.githubusercontent.com/ros/rosdistro/master/index-v4.yaml

  humble:
    distribution: [humble/distribution.yaml]
    distribution_cache: http://repo.ros2.org/rosdistro_cache/humble-cache.yaml.gz
    distribution_status: active
    distribution_type: ros2
    python_version: 3

http://repo.ros2.org/rosdistro_cache/humble-cache.yaml.gz <- this cache is pointing to 2.5.4-1 of MoveIt

    moveit: {doc: {type: git, url: 'https://github.com/ros-planning/moveit2.git', version: main}, release: {packages: [chomp_motion_planner, moveit, moveit_chomp_optimizer_adapter, moveit_common, moveit_configs_utils, moveit_core, moveit_hybrid_planning, moveit_kinematics, moveit_planners, moveit_planners_chomp, moveit_planners_ompl, moveit_plugins, moveit_resources_prbt_ikfast_manipulator_plugin, moveit_resources_prbt_moveit_config, moveit_resources_prbt_pg70_support, moveit_resources_prbt_support, moveit_ros, moveit_ros_benchmarks, moveit_ros_control_interface, moveit_ros_move_group, moveit_ros_occupancy_map_monitor, moveit_ros_perception, moveit_ros_planning, moveit_ros_planning_interface, moveit_ros_robot_interaction, moveit_ros_visualization, moveit_ros_warehouse, moveit_runtime, moveit_servo, moveit_setup_app_plugins, moveit_setup_assistant, moveit_setup_controllers, moveit_setup_core_plugins, moveit_setup_framework, moveit_setup_srdf_plugins, moveit_simple_controller_manager, pilz_industrial_motion_planner, pilz_industrial_motion_planner_testutils], tags: {release: 'release/humble/{package}/{version}'}, url: 'https://github.com/ros2-gbp/moveit2-release.git', version: 2.5.4-1}, source: {test_commits: false, test_pull_requests: false, type: git, url: 'https://github.com/ros-planning/moveit2.git', version: main}, status: developed}
    moveit_msgs: {doc: {type: git, url: 'https://github.com/ros-planning/moveit_msgs.git', version: ros2}, release: {tags: {release: 'release/humble/{package}/{version}'}, url: 'https://github.com/ros2-gbp/moveit_msgs-release.git', version: 2.2.2-1}, source: {type: git, url: 'https://github.com/ros-planning/moveit_msgs.git', version: ros2}, status: developed}
    moveit_resources: {doc: {type: git, url: 'https://github.com/ros-planning/moveit_resources.git', version: ros2}, release: {packages: [moveit_resources, moveit_resources_fanuc_description, moveit_resources_fanuc_moveit_config, moveit_resources_panda_description, moveit_resources_panda_moveit_config, moveit_resources_pr2_description], tags: {release: 'release/humble/{package}/{version}'}, url: 'https://github.com/ros2-gbp/moveit_resources-release.git', version: 2.0.6-1}, source: {type: git, url: 'https://github.com/ros-planning/moveit_resources.git', version: ros2}, status: developed}
    moveit_visual_tools: {doc: {type: git, url: 'https://github.com/ros-planning/moveit_visual_tools.git', version: ros2}, release: {tags: {release: 'release/humble/{package}/{version}'}, url: 'https://github.com/ros2-gbp/moveit_visual_tools-release.git', version: 4.1.0-1}, source: {type: git, url: 'https://github.com/ros-planning/moveit_visual_tools.git', version: ros2}, status: maintained}

[moriarty]

https://github.com/ros/rosdistro/commit/b9eb03ef0db05d22643de4fea9deb6e63c382e5b

https://github.com/ros/rosdistro/blob/8d667705ed7db41b1e203343f69e3d4fcbb23743/humble/distribution.yaml#L2988

https://github.com/ros-planning/moveit2/pull/1849

The fix is in the humble branch but it hasn't been released

@tylerjw has opened a pr https://github.com/ros/rosdistro/pull/37267 reverting the moveit_msgs bump in humble