Rosdep only shows the last resolution error for each package

[rotu]

I'm building ros2_ouster from source on Ubuntu Focal against ROS2 Master, which depends on pcl_conversions and libpcl-all

While rosdep shows a warning for libpcl-all, it does not show a warning for pcl_conversions:

rosdep install --from src/ros2_ouster_drivers -ir
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
ros2_ouster: No definition of [libpcl-all] for OS version [focal]
Continuing to install resolvable dependencies...
#All required rosdeps installed successfully

It's also missing from rosdep check:

rosdep check --from src/ros2_ouster_drivers -ir                   
All system dependencies have been satisfied
ERROR[ros2_ouster]: No definition of [libpcl-all] for OS version [focal]
    rosdep key : libpcl-all
    OS name    : ubuntu
    OS version : focal
    Data: arch:
- pcl
debian:
  buster:
  - libpcl-apps1.8
...
  bionic:
  - libpcl-apps1.8

Spoofing my os version to bionic causes the missing key to show up:

rosdep check --from src/ros2_ouster_drivers --os=ubuntu:bionic -ir
System dependencies have not been satisfied:
apt libpcl-apps1.8
apt libpcl-common1.8
apt libpcl-features1.8
apt libpcl-filters1.8
apt libpcl-io1.8
apt libpcl-kdtree1.8
apt libpcl-keypoints1.8
apt libpcl-ml1.8
apt libpcl-octree1.8
apt libpcl-outofcore1.8
apt libpcl-people1.8
apt libpcl-recognition1.8
apt libpcl-registration1.8
apt libpcl-sample-consensus1.8
apt libpcl-search1.8
apt libpcl-segmentation1.8
apt libpcl-stereo1.8
apt libpcl-surface1.8
apt libpcl-tracking1.8
apt libpcl-visualization1.8
ERROR[ros2_ouster]: Cannot locate rosdep definition for [pcl_conversions]
    rosdep key : pcl_conversions
    OS name    : ubuntu
    OS version : bionic
    Data: <no data>

[rotu]

@cottsay, take a gander. Are my expectations wrong? @SteveMacenski, FYI

[cottsay]

First, I'll note that pcl_conversions is a ROS package. It was released for Eloquent, but like other packages, hasn't been released yet for Foxy. So we're not expecting to see a rosdep key for it, but since we don't differentiate whether dependencies are expected to be a ROS package or a rosdep key at declaration time, rosdep doesn't know to display a different message for the missing package.

I think the behavior you're seeing is that only the first last missing dependency for a package is reported. A rosdep rule for libpcl-all is missing for Focal, which is why you're seeing a difference in behavior.

With the difference in behavior explained, there are two things that need to happen:

1. Add a rosdep rule for Ubuntu Focal for the libpcl-all key.
2. Include pcl_conversions in your workspace so that the -i argument resolves (ignores) the dependency.

An improvement to clarify this situation would be to continue attempting to resolve rosdep keys even after a resolution failure occurs aggregate all resolution errors, so that you get all of the resolution failures for a package instead of only the first last one.

[rotu]

Why is only the first missing dependency reported? Shouldn't -r path make that not the case?

[cottsay]

I would assume that -r causes installation of successfully resolved dependencies to occur despite the presence of errors, and it does not influence the behavior that resolution of dependencies in a package stops when a resolution error occurs.

I'm making a guess at this point, though. We should look at the code.

[cottsay]

Could you try running rosdep install --from src/ros2_ouster_drivers -ir --skip-keys "libpcl-all" on your Focal machine and see if the pcl_conversions error shows up?

[cottsay]

I confirmed this myself. Here is the spot where the problem occurs: https://github.com/ros-infrastructure/rosdep/blob/cfa21c89d440e3fcb6c41b1f4a5f4947a5649b41/src/rosdep2/lookup.py#L420-L423

While I was on the right track in describing the behavior, I missed the mark on the cause. Resolution happens for every key in the package, but only the last error gets reported. Seems like a great opportunity for a PR that makes the behavior more expected.

[rotu]

Yeah and adding libpcl-all to the rosdep .list file makes it so the dependency shows up up because it changes the nature of the problem from a resolution error to a missing dependency:

rosdep check --from src/ros2_ouster_drivers -ir
System dependencies have not been satisfied:
apt libpcl-apps1.10
apt libpcl-common1.10
apt libpcl-features1.10
apt libpcl-filters1.10
apt libpcl-io1.10
apt libpcl-kdtree1.10
apt libpcl-keypoints1.10
apt libpcl-ml1.10
apt libpcl-octree1.10
apt libpcl-outofcore1.10
apt libpcl-people1.10
apt libpcl-recognition1.10
apt libpcl-registration1.10
apt libpcl-sample-consensus1.10
apt libpcl-search1.10
apt libpcl-segmentation1.10
apt libpcl-stereo1.10
apt libpcl-surface1.10
apt libpcl-tracking1.10
apt libpcl-visualization1.10
ERROR[ros2_ouster]: Cannot locate rosdep definition for [pcl_conversions]
    rosdep key : pcl_conversions
    OS name    : ubuntu
    OS version : focal
    Data: <no data>

[rotu]

FYI, @cottsay: Also posted https://answers.ros.org/question/348230/how-do-i-use-rosdep-install/. I'm missing some very foundational knowledge about this tool.

[SteveMacenski]

@rotu so let me summarize what my understanding here is:

• 1 You can't find PCL / perception on rosdep master
• 2 There's an error with how rosdep gives you back information
• 3 Your ticket is named after the rosdep info back error, so you really don't need help with 1, but we're talking about it anyway?

1 is easy, its not released or the key doesn't exist for the OS, you can easily submit a PR for it if the binaries are being built for it. If not, poking maintainers to release the binraries so then you can update the rosdep list. Ok, done.

2 and 3 sound like rosdep issues, no particular action item for me?

[ShivamPR21]

@rotu how did you solved it for pcl_conversiosns/perception_pcl I am having the almost same problem but this time my personal system is working fine rosdep is able to find pcl_conversiosns/perception_pcl but on CI system I am having the same problem

and providing error:-

ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
nav2_map_server: Cannot locate rosdep definition for [perception_pcl]

On my personal system:-

rosdep install --simulate --reinstall  --from-paths src --ignore-src --rosdistro foxy

is able to find them

.....
sudo -H apt-get install ros-foxy-perception-pcl
sudo -H apt-get install ros-foxy-pcl-conversions
.....

but on circle ci it's not working.

[rotu]

@SteveMacenski, correct. This is not particular to ouster or PCL stuff; they are just where I saw this behavior.

@ShivamPR21 I can’t say for certain. It does look like the foxy release for perception_pcl was yesterday and the CI run was 3 days ago, so I’d recommend just rerunning and seeing if your CI works now.

[ShivamPR21]

@ShivamPR21 I can’t say for certain. It does look like the foxy release for perception_pcl was yesterday and the CI run was 3 days ago, so I’d recommend just rerunning and seeing if your CI works now.

Nope, it's still not working.