Rosdep typo in custom yaml can permanently corrupted depedency name

[fprott]

I found an interesting and pretty serious bug under ROS1 Noetic (Ubuntu 20.04.6 LTS, Python 3.8.10).

Basically I manged to permanently corrupt the name of a dependency.

I wanted to build myself a yaml file to handle a lot of dependencies in a structured manner. Most of my dependencies where apt-get related, however one was pip related. I forgot that so my file looked like that.

python-shapely:
  ubuntu:
    packages: [shapely]

I run the file and noticed the error right away. So I stopped rosdep while it was running and corrected my mistake.

python-shapely:
  ubuntu:
    pip:
      packages: [shapely]

No big deal, right? Nope! No matter what I do or what I do the dependency python-shapely will only and always trigger the apt-get install command as well as the following error

executing command [sudo -H apt-get install -y python-shapely]
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package python-shapely
ERROR: the following rosdeps failed to install
  apt: command [sudo -H apt-get install -y python-shapely] failed
  apt: Failed to detect successful installation of [python-shapely]

Basically the name has become toxic. If I use any other name the system works as intended! So if you try:

python-shapely: #breaks everything
  ubuntu:
    pip:
      packages: [shapely]

python-example: #works like a charm
  ubuntu:
    pip:
      packages: [shapely]

I tried to delete the cache in .ros/rosdep/ by deleting the folder. I tried to reinstall the rosdep tool. Heck, I even restated my machine for the first time in forever. I even searched trough every file mentioning python-shapely, but the freaking thing is saved no where. I tried everything, the name is just broken.

So, any way to fix this? Where the heck is the cache responsible for that?

[cottsay]

There is already an entry for python-shapely in the rosdep db: https://github.com/ros/rosdistro/blob/844ffe8d9962bd4a304e5722ac612aac9ea0ceff/rosdep/python.yaml#L3821-L3829

Correct me if I'm wrong, but my understanding is that you're creating a supplementary rosdep database and adding it to sources.list.d. Off the top of my head, I'm not sure how rosdep would handle a redefinition like this, but my guess is that it would merge the rules.

[cottsay]

Some quick tests show that the first definition appears to win, so try ensuring that your rules get processed before the official rules by either naming the file in sources.list.d higher alphabetically or listing your yaml file further up in 20-default.list.

[tfoote]

The order of rosdep sources matters. The first match will be used. I suspect that your custom rule was never activated because it's being shadowed by the main one.

I you move your sources list earlier then you should be able to override it.

[cottsay]

jinx!

[fprott]

There is already an entry for python-shapely in the rosdep db: https://github.com/ros/rosdistro/blob/844ffe8d9962bd4a304e5722ac612aac9ea0ceff/rosdep/python.yaml#L3821-L3829

Correct me if I'm wrong, but my understanding is that you're creating a supplementary rosdep database and adding it to sources.list.d. Off the top of my head, I'm not sure how rosdep would handle a redefinition like this, but my guess is that it would merge the rules.

You are fully correct. The freaking thing is already in a list that is loaded via a URL. And that long list is just wrong at this point. Somebody placed an there even tough it is not available from apt-get. Apparently the is always interpreted as "try first to get it via apt-get and if that fails don't try anything else". I already made a pull request to fix this.

Some quick tests show that the first definition appears to win, so try ensuring that your rules get processed before the official rules by either naming the file in sources.list.d higher alphabetically or listing your yaml file further up in 20-default.list.

Got it! I will fix the order then!

Many thanks for the quick and correct help. This is a great community! I am pretty sure this was the sole root cause of this problem. So I will close this.