test_depend in package.xml is not installed

[annaborn]

Hello Sylvain @doudou!

we are using autoproj in ros workspace. There is one issue with test_depend in package.xml. They will be not installed per default or even after enabling the tests with autoproj.

In ros_package_manifest.rb https://github.com/rock-core/autoproj/blob/e10dd92490b528638e5173da0c0636313cba73cd/lib/autoproj/ros_package_manifest.rb#L96 the test mode will be set for test_depend packages. So these dependencies will be not installed per default. However, I can not enable the test with "autoproj test enable" for ros packages, since no task block is assigned in the test Autobuild::Utility.

#<Autobuild::TestUtility:0x000059e2ca9ff8b0 @name="test", @task=nil, @package=#<Autobuild::ImporterPackage name=...>, @available=true, @enabled=true, @source_ref_dir=nil, @source_dir=nil, @target_dir=nil, @install_on_error=true, @no_results=false, @invoked=false, @success=false, @installed=false, @coverage_enabled=nil, @coverage_source_dir=nil, @coverage_target_dir=nil>

I am not sure, what task block I should add to ros packages. I took a look in the definition of catkin_package, and did not find any definition for that, since it is pure import_package.

What would be your suggestion to make the test dependencies work with autoproj? One idea is to comment the mode settings in ros_package_manifest.rb out. So any dependency incl. test dependency will be installed per default.

[doudou]

The test block is meant to run the tests ... how do you run tests in catkin packages ?

[g-arjones]

I took a look in the definition of catkin_package, and did not find any definition for that, since it is pure import_package

That's not right... A catkin_package is by default just a cmake_package with a package.xml instead of a manifest.xml: https://github.com/rock-core/autoproj/blob/e10dd92490b528638e5173da0c0636313cba73cd/lib/autoproj/autobuild_extensions/dsl.rb#L282-L287

The behavior you are seeing is correct. Test dependencies are only installed if the package both has tests and they are enabled.

If you are overriding the catkin_package helper in your configuration to be an import_package instead then you have to add a test task yourself:

import_package "my_package" do |pkg|
    pkg.test_utility.no_results = true  # if your tests generate results then you may set `pkg.test_utility.source_dir` instead
    pkg.test_utility.task do
        pkg.progress_start "running tests for %s", done_message: "successfully ran tests for %s" do
            cmd = %w[my_test_command --argument]
            pkg.run("test", *cmd, working_directory: pkg.srcdir)
        end
    end
end

If you want autoproj to install the dependencies regardless and not run any tests then just add an empty block.

[annaborn]

@doudou @g-arjones we are not using catkin_package, since we are working with ros2 and using colcon to build the packages. A new package type colcon_package as import_package was introduced in the package_set. https://github.com/dfki-ric/ros2-package_set/blob/master/lib/colcon_package.rb. So the autoproj is used only for managing the workspace: updating the packages, installing all dependencies and etc.

def colcon_package(name, type = :import_package, move_to: name, workspace: Autoproj.workspace)
    send(type, name) do |pkg|
        pkg.use_package_xml = true
        yield(pkg) if block_given?
    end
    if name != move_to then
        move_package name, move_to
    end
end

@g-arjones Gabriel, thank you for the information. Yes, you are totally right. I mixed the package type definition. Thank you for your suggestion to add an empty block to the colcon_package. I will try this out.

Do you have experience or work to integrate autoproj with ros2?

[g-arjones]

Do you have experience or work to integrate autoproj with ros2?

Yes. I have been using autoproj with ros2 for years now but I don't use colcon at all (autoproj calls cmake directly). The ament cmake macros generate dsv files which can be parsed by autoproj to generate its own environment (in the workspace for ament packages but also recursively on /opt/ros/<distro>). From there, everything else is pretty seamless.

[annaborn]

@g-arjones Gabriel, there was no email address in your github profil. I hope, it is ok, if I write my questions regarding autoproj+ros2 here.

What package type definition do you use for the ros2 package to build them with autoproj? Does it require some autoproj extension, so the autoproj can handle ament/colcon packages?

In github I found just one example for using ros2+autoproj. It is the Open Rise Robotics project https://github.com/orise-robotics, where the autobuild was extended with the configuration and build steps for ament und colcon packages. I did test it yet.

[g-arjones]

Just a small wrapper around cmake_package:

def ament_cmake_package(name, workspace: Autoproj.workspace)
    cmake_package(name, workspace: Autoproj.workspace) do |pkg|
        pkg.use_package_xml = true
        pkg.post_import do
            pkg.define "BUILD_TESTING", !!pkg.test_utility.enabled?
            pkg.test_utility.source_dir = File.join(pkg.builddir, "test_results", pkg.description.name)
            pkg.define "AMENT_TEST_RESULTS_DIR", File.join(pkg.builddir, "test_results")
        end

        pkg.extend ROS2::AmentEnvironment
        yield(pkg) if block_given?
    end
end

The ROS2::AmentEnvironment is an extension providing an #update_environment method that loads the dsv files I mentioned in my previous comment. Since you are probably sourcing the ament-generated env files (i.e. /opt/ros/humble/setup.bash and install/setup.bash) and not using Autoproj.isolate_environment, that part could be skipped.

[annaborn]

Thank you a lot, Gabriel! I will try it out now.

What is about ros2 python package? Can it be defined as python_package or do I need extra definition for this package type? If we would use isolated environment, where I can find ROS2::AmentEnvironment implementation?

Do you plan to add the ament_cmake_package definition, ROS2::AmentEnvironment as well other extensions directly into autoproj?

@g-arjones

[g-arjones]

What is about ros2 python package? Can it be defined as python_package or do I need extra definition for this package type?

For ament python packages you just have to setup a test task and either no_results or source_dir on the test utility if you want test support. Other than that, just pkg.use_package_xml = true.

If we would use isolated environment, where I can find ROS2::AmentEnvironment implementation?

You do have to override #update_environment to load the dsv files if you want to use isolated environments and/or separate prefixes. Unfortunately, that's not something I can share right now but it's not difficult to implement. The format is really easy to parse.

Do you plan to add the ament_cmake_package definition, ROS2::AmentEnvironment as well other extensions directly into autoproj?

Not at this point, sorry.

[annaborn]

Thank you a lot for your help, Gabriel! I already build all ament cmake und python packages in the workspace and will write the loader for dsv files. :)

Maybe in some future, we will see the ros2 integration as a part of autoproj :)

I will close this issue.