search

orocos-toolchain / orogen

Code generator for components and type handling in Rock - the Robot Construction Kit - and the Orocos Toolchain
http://rock-robotics.org
Other
4 stars 35 forks source link

manifest.xml: switch metaruby to tag type <rosdep> #140

Closed meyerj closed 3 years ago

meyerj commented 4 years ago

… such that it can be installed correctly by rosdep:

Without this patch:

$ rosdep install -y --from-path . --ignore-src
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
orogen: Missing resource metaruby
ROS path [0]=/build/orocos_toolchain

With this patch (on Ubuntu 18.04):

$ rosdep install -y --from-path . --ignore-src
executing command [sudo -H gem install metaruby]
Fetching: rexml-3.2.4.gem (100%)
Successfully installed rexml-3.2.4
Fetching: kramdown-2.3.0.gem (100%)
Successfully installed kramdown-2.3.0
Fetching: utilrb-3.1.0.gem (100%)
Successfully installed utilrb-3.1.0
Fetching: metaruby-2.0.0.gem (100%)
Successfully installed metaruby-2.0.0
Parsing documentation for rexml-3.2.4
Installing ri documentation for rexml-3.2.4
Parsing documentation for kramdown-2.3.0
Installing ri documentation for kramdown-2.3.0
Parsing documentation for utilrb-3.1.0
Installing ri documentation for utilrb-3.1.0
Parsing documentation for metaruby-2.0.0
Installing ri documentation for metaruby-2.0.0
Done installing documentation for rexml, kramdown, utilrb, metaruby after 2 seconds
4 gems installed
#All required rosdeps installed successfully

This patch effectively reverts 8cb82a57d42942c8a6029e043389f84c2e737848.

Background information: Unfortunately rosdep still prefers manifest.xml (aka dry) before package.xml (aka wet) if it exists. In manifest.xml packages listed with <depend> must be valid ROS packages with a package.xml or manifest.xml file. In contrast, <rosdep> dependencies are looked up in the rosdep database. rosdep also supports packages of type gem (cf. rosdep2/platforms/gem.py) and metaruby is available as a gem. So with this patch and after key metaruby has been added to the rosdep database of ruby packages, it is possible to install all dependencies of an orocos_toolchain workspace with rosdep again for source builds.

For binary releases into ROS orogen cannot depend on gems, and metaruby must be provided as a ROS package, too. That is what we did in 2014 in repo metaruby-release for the indigo release and what triggered the change in 8cb82a57d42942c8a6029e043389f84c2e737848. But at the moment there are no binary releases in ROS and for source builds using the gem for metaruby is more convenient. If there would be binary ROS releases again in the future, it is possible to delete or rename manifest.xml in the release repository, such that bloom and other tools only consider the dependencies in package.xml, where there is no distinction between ROS packages and other dependencies found via rosdep.

As far as I know for Autoproj it does not matter whether dependencies are listed using tag <rosdep> or <depend>, correct?

In https://github.com/orocos-toolchain/rtt/pull/321 it has been discussed to eventually drop manifest.xml completely and switch to autoproj.xml to get rid of the ROS/Autoproj compatibility issues.

In the second commit, https://github.com/orocos-toolchain/orogen/pull/140/commits/7a294614dc6e1034eb2aeadc434ce1ff25196307, I updated the maintainer email in package.xml. The domain lists.mech.kuleuven.be does not work anymore since the department switched to a centralized list server infrastructure at https://ls.kuleuven.be a few years ago. orocos-dev@orocos.org currently forwards to orocosdev@ls.kuleuven.be.

doudou commented 4 years ago

This should be transparent for Autoproj, yes. It's all green on CI, so that's good for us.

However, this is picking up the released version of metaruby. Even though we're trying to cleanup our act version-wise, I currently dont't guarantee that orogen master will work on metaruby gem.

meyerj commented 4 years ago

However, this is picking up the released version of metaruby. Even though we're trying to cleanup our act version-wise, I currently dont't guarantee that orogen master will work on metaruby gem.

Thanks for the warning. I am going to check whether the tests pass with the latest metaruby (and utilrb?) versions from https://rubygems.org/.

Do you propose another way to provide a specific version of metaruby for a specific version of orogen, for example to add tools-metaruby as a submodule to orocos_toolchain or to this repo? How does Autoproj know which version to install?

On the other hand, if the versions from https://rubygems.org/ work, would you consider to remove submodule utilrb? If I understand correctly, it is not and was never really part of "the Orocos toolchain", but a dependency of orogen.

doudou commented 4 years ago

Do you propose another way to provide a specific version of metaruby for a specific version of orogen, for example to add tools-metaruby as a submodule to orocos_toolchain or to this repo? How does Autoproj know which version to install?

I, and by extension Rock, has sucked at maintaining versioning. We basically have been running from master for the last few years. I'm starting an effort to properly version things, at which point it would be easier to answer this question. The closest you could get to this "workflow" is indeed to use submodules. On the Autoproj side, releases are handled by producing a consistent version file, which is usually a snapshot of the current state.

would you consider to remove submodule utilrb?

I'm not involved in the orocos-toolchain maintenance, so this is your call. But then, utilrb has the same issues than metaruby concerning "compatibility". FYI, the codegen part of the orogen test suite has been too hard to maintain, I nowadays rely on a separate Rock test suite (https://github.com/rock-core/rock.build-tests-buildconf) for everything that requires builds. I would suggest you start to do the same for the typekit-only codegen usage of orogen. The internals (non-codegen) parts is unit-tested directly.

dustingooding commented 3 years ago

I was using this branch with success in a local from-source build. I've now removed the use of this branch (as it's deleted) and am now getting the error related to missing metaruby.

ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
orogen: Missing resource metaruby
ROS path [0]=/root/orocos_ws/orocos_toolchain

Maybe the solution is to merge these commits into toolchain-2.9 branch?

meyerj commented 3 years ago

Maybe the solution is to merge these commits into toolchain-2.9 branch?

Yes, likely, if you use the docker image for Orocos and ROS 2 in Ubuntu Focal built from https://github.com/orocos/orocos-docker-images/tree/master/ros2/ubuntu. There we define the rosdep keys facets, metaruby and utilrb in prereqs.yaml. I actually forgot that they are not yet available in the rosdep database at https://github.com/ros/rosdistro and opened a pull request: https://github.com/ros/rosdistro/pull/28155

If that would be accepted, then prereqs.yaml could be removed for the Orocos docker image.

Eventually you do not need orogen, utilrb and metaruby, if you only want to use RTT and rtt_ros2_integration? Currently they are submodules of orocos_toolchain, and that is why rosdep tries to satisfy dependencies. But at least with https://github.com/orocos-toolchain/orocos_toolchain/pull/37 and on the ros2 branch BUILD_OROGEN is OFF by default, because I could not get it to work last summer in newer Ubuntu versions. Other users seem to have similar issues: https://github.com/orocos-toolchain/orocos_toolchain/issues/39

If someone with more insights in the Ruby world can fix the issues with rosdep and with running orogen and typegen itself, and support it for ROS and orocos_toolchain users, that would be nice. Otherwise I suggest to remove the submodules starting from the next release toolchain-2.10 and on branch master. Of course it is still possible to add orogen and its dependencies to the catkin/colcon workspace in any other way, provided that building with CMake will still be supported in the future and that the package.xml files and env-hooks are kept up-to-date.