[question] [ROS 2] what type of packages should be detected by rosdep?

[mikaelarguedas]

Follow up of https://github.com/ros-infrastructure/rosdep/pull/699

Should rosdep detect any package with a package.xml file or only packages registered in the ament index?

Currently it successfully detects packages that are registered in the ament index but not packages that only have a package.xml

[tfoote]

rosdep can work with packages that only have a package.xml but it needs to know where to find them with the --from-path argument.

rosdep install --from-path src for a ROS 2 workspace with just the vcs checkout without any build artifacts (ament index)

[mikaelarguedas]

I see now that 6months later the question is unclear without extra context. The use case here was mostly about detecting packages in underlays when running rosdep install --from-paths on an overlay. So passing --from-paths or not will not be impacting the underlying issue.

Currently any package in the underlay not registered in the ament index, even if it is on the AMENT_PREFIX_PATH and installs a package.xml, will not be detected by rosdep and rosdep will try to install it from binary.

This has been causing issues when building overlays in general and especially when trying to build on top of the nightly archive. The current way to work around it is to add the underlay to the ROS_PACKAGE_PATH so that these packages are detected by rosdep. This doesnt seem like a good long term solution for ROS 2 (as it's based on the deprecated ROS_PACKAGE_PATH).

This makes the story of building on top of the nightly archive or building overlays in general pretty unpleasant, especially if trying to integrate third-party packages. A more in-depth description is available at https://github.com/ros2/tinyxml_vendor/pull/14#issuecomment-566719596.

The hope of this ticket is to build consensus that detecting all packages installing a package.xml on the AMENT_PREFIX_PATH should be detected by rosdep providing a more consistent workspace chaining experience and lower barrier of entry to adding third-party packages.

[Levi-Armstrong]

We are looking for an option to support pure cmake packages within the ROS2 ecosystem. Switching to ament is currently not an option which requires the user to manually update the AMENT_PREFIX_PATH. We are currently writing the ament index file but not sure how to solve the AMENT_PREFIX_PATH. This is also an issue with using pluginlib to load plugins from a ROS2 packages when the plugin is located in a pure cmake package.

[dirk-thomas]

@Levi-Armstrong I haven't tried it but the following should work (as of Eloquent, not in Dashing). Generate and install the following two files:

• share/<pkgname>/package.dsv:

  source;share/<pkgname>/hook/ament_prefix_path.dsv

• share/<pkgname>/hook/ament_prefix_path.dsv:

  prepend-non-duplicate;AMENT_PREFIX_PATH;

[Levi-Armstrong]

Thank you for the quick response we will give it a try and report back.

[Levi-Armstrong]

@dirk-thomas I ran into one issue. It looks like the package.dsv gets created by something else during the build over writing the package.dsv I created. I did confirm after everything is built if I open the package.dsv and add the source;share/${PROJECT_NAME}/hook/ament_prefix_path.dsv to the file and then source that my pure cmake package now shows up in the AMENT_PREFIX_PATH environment variable. Any ideas how to solve the package.dsv issue?

Commands added to the pure CMake package.

file(WRITE ${CMAKE_INSTALL_PREFIX}/share/ament_index/resource_index/packages/${PROJECT_NAME} "")
file(WRITE ${CMAKE_INSTALL_PREFIX}/share/${PROJECT_NAME}/hook/ament_prefix_path.dsv "prepend-non-duplicate;AMENT_PREFIX_PATH;")
file(APPEND ${CMAKE_INSTALL_PREFIX}/share/${PROJECT_NAME}/package.dsv "source;share/${PROJECT_NAME}/hook/ament_prefix_path.dsv")

[dirk-thomas]

It looks like the package.dsv gets created by something else during the build over writing the package.dsv I created.

Instead of naming the file package.dsv can try naming it local_setup.dsv?

[Levi-Armstrong]

I switched to the local_setup.dsv and the file gets created, but AMENT_PREFIX_PATH is not being updated.

[Levi-Armstrong]

I believe Colcon is the one creating the package.dsv, so could it recognize that the package is a cmake package and do what you suggested by creating the the index file, ament_prefix_path.dsv and append the ament_prefix_path.dsv to the package.dsv?

colcon list

ament_pkg   src/ROSDepISSUE/ament_pkg   (ros.ament_cmake)
cmake_pkg   src/ROSDepISSUE/cmake_pkg   (ros.cmake)
opw_kinematics  src/opw_kinematics  (ros.cmake)

[dirk-thomas]

Can you shate the CMake package so I can give it a try?

[Levi-Armstrong]

Sure. Here is the example workspace I have been working with. You can remove the requirement for opw_kinematics since it is not required for the purpose of testing this since I created a simple cmake package.

[dirk-thomas]

I had to check for myself and until now it isn't possible for a plain CMake package or ROS CMake packages not using ament / catkin. Please try the changes from colcon/colcon-ros#107 as well as the referenced PR colcon/colcon-core#357 together they should enable this use case.

Your CMake logic needs to install env hooks into the share/<pkgname>/env_hook for this to work.

Btw you should file(WRITE ${CMAKE_INSTALL_PREFIX}/...) but use the install() function instead:

file(
  WRITE ${CMAKE_CURRENT_BINARY_DIR}/ament_prefix_path.dsv
  "prepend-non-duplicate;AMENT_PREFIX_PATH;")
install(
  FILES ${CMAKE_CURRENT_BINARY_DIR}/ament_prefix_path.dsv
  DESTINATION share/${PROJECT_NAME}/env_hook)

[dirk-thomas]

I had to check for myself and until now it isn't possible for a plain CMake package or ROS CMake packages not using ament / catkin. Please try the changes from colcon/colcon-ros#107 as well as the referenced PR colcon/colcon-core#357 together they should enable this use case.

Forget what I said before. There is already a documented way to register custom environment hooks. Please see the hooks entry in colcon.pkg files.

[Levi-Armstrong]

@dirk-thomas The links colcon.pkg worked. Do you see any issues with what I have done?

Added the following to the CMakeLists.txt

file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/share/ament_index/resource_index/packages/${PROJECT_NAME} "")
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/share/ament_index/resource_index/packages/${PROJECT_NAME} DESTINATION share/ament_index/resource_index/packages)

file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/share/${PROJECT_NAME}/hook/ament_prefix_path.sh "export AMENT_PREFIX_PATH=${CMAKE_INSTALL_PREFIX}:$AMENT_PREFIX_PATH")
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/share/${PROJECT_NAME}/hook/ament_prefix_path.sh DESTINATION share/${PROJECT_NAME}/hook)

Added the colcon.pkg file to my pure cmake package with the following content.

{
    "hooks": ["share/cmake_pkg/hook/ament_prefix_path.sh"]
}

[Levi-Armstrong]

I assume the colcon.pkg must be installed; is that correct?

[dirk-thomas]

Do you see any issues with what I have done?

Your .sh file has two downsides:

• it contains an absolute path which makes your installed package not relocatable and
• it doesn't work on e.g. Windows where you would need a .bat or .ps1 file.

Both problems can be avoided by using the approach of the .dsv file from my previous comment.

I assume the colcon.pkg must be installed; is that correct?

No, it doesn't need to be installed. It is only used by colcon at build time of the package.

[Levi-Armstrong]

@dirk-thomas Thank you for the help. The following below worked.

Added the following to the CMakeLists.txt

file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/share/ament_index/resource_index/packages/${PROJECT_NAME} "")
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/share/ament_index/resource_index/packages/${PROJECT_NAME} DESTINATION share/ament_index/resource_index/packages)

file(WRITE ${CMAKE_CURRENT_BINARY_DIR}/share/${PROJECT_NAME}/hook/ament_prefix_path.dsv "prepend-non-duplicate;AMENT_PREFIX_PATH;")
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/share/${PROJECT_NAME}/hook/ament_prefix_path.dsv DESTINATION share/${PROJECT_NAME}/hook)

Added the colcon.pkg file to my pure cmake package with the following content.

{
    "hooks": ["share/cmake_pkg/hook/ament_prefix_path.dsv"]
}

[Levi-Armstrong]

@dirk-thomas We ran into another situation. The above worked when building the pure cmake package within a ROS2 workspace but when building it using colcon in a ROS1 workspace leveraging catkin we had to create similar hooks for ROS_PACKAGE_PATH. Does colcon provide any way for a cmake package to detect if catkin or ament is being use within the workspace? Also just curious can you mix package using catkin and ament in the same workspace? If not we can install both environment hooks but wanted to check.

[dirk-thomas]

Does colcon provide any way for a cmake package to detect if catkin or ament is being use within the workspace?

colcon doesn't provide anything special here. Depending on your scenario the package can probably infer that from the environment - either based on environment variables like ROS_VERSION or by trying to find both assuming only one of them can be found.

Also just curious can you mix package using catkin and ament in the same workspace?

From a technical point of view there is no difference between them. colcon just finds all packages and triggers each packages build according to the determine build type and while honoring dependencies.

But beyond that technical aspect a catkin-based package commonly tries to use other catkin-based packages and uses catkin specific CMake functions. Some for ament packages. If the names don't collide they usually easily coexists - but their CMake logic is commonly not able to mix between the types (a catkin package depending on / using an ament package or vice versa).