Conflicting pkg-config files

[meyerj]

For most Orocos packages using catkin now (typekits, libraries, etc.) cmake generates and installs two conflicting versions of pkg-config files.

Example:

# generated by Orocos
$ cat /opt/orocos/hydro/lib/pkgconfig/rtt_rosnode-gnulinux.pc 
prefix=/opt/orocos/hydro
libdir=${prefix}/lib
includedir=${prefix}/include/orocos
orocos_libdir=${libdir}/orocos/gnulinux/rtt_rosnode

Name: rtt_rosnode-gnulinux
Description: rtt_rosnode-gnulinux package for Orocos
Requires: orocos-rtt-gnulinux 
Version: 1.0
Libs:  -L${orocos_libdir}/plugins  -lrtt_rosnode-gnulinux
Cflags: -I${includedir}

# generated by catkin
$ cat /opt/orocos/hydro/lib/pkgconfig/rtt_rosnode.pc 
prefix=/opt/orocos/hydro

Name: rtt_rosnode
Description: Description of rtt_rosnode
Version: 2.6.0
Cflags: 
Libs: -L/opt/orocos/hydro/lib 
Requires: roscpp

The existence of the file generated by catkin also breaks orocos_use_package() as the filename without the OROCOS-TARGET suffix is tried first. Is it possible to swap these two in UseOrocosRTT-helpers.cmake#L121?

Or is there a better solution, e.g. to add ${PROJECT_NAME}-${OROCOS_TARGET} to the DEPENDS list in the catkin_package() call so that pkg-config pulls in the libraries recursively?

[jbohren]

Yeah, this is something I've been putting off, mostly because I wanted to get feedback from @smits. Looks like now is a good time to start talking about it, at least.

So, to review the different buildsystems and the crud they do to the system:

Catkin

• calling find_package(catkin COMPONENTS ...) will do two things for each package:
  1. it will execute any dependency's CFG_EXTRAS (CMake code)
  2. it will define a bunch of variables based on the arguments to each dependency's catkin_package() call
• Calling the catkin_package() macro generates a CMake config file (actionlibConfig.cmake) and a pkg-config file (actionlib.pc) for packages which depend on this package. Now, the pkg-config files appear to contain the same information as the CMake config files, and based on REP-0136 it appears the only purpose of these pkg-config files is so that rosbuild packages can build against catkin-based packages.

Orocos

• Calling the orocos_component()/orocos_plugin()/etc macros constructs lists of linker args for each type of library
• Calling the orocos_generate_package() macro generates a pkg-config file which lists all of the shared objects (libraries, typekits, plugins, components) as link libraries
• Calling the orocos_use_package(...) macro sets variables similarly to the way Catkin does (i.e. normal find_package() behavior), but it will also automatically link all targets against all depedency libraries by default unless ${OROCOS_NO_AUTO_LINKING} is set to true.

However! When you include UseOROCOS-RTT.cmake, it automatically calls orocos_use_package() on every package manifest dependency (see here). Previously, this read the manifest.xml, and currently, this will read the "package.xml" file in the ${CMAKE_CURRENT_SOURCE_DIR} and use xpath to get all <build_depend> tags in the package.xml file (see here).

This last bit might actually be undesirable in this brave new catkin world, and if someone happens to both call orocos_generate_package() along with catkin_package() and exported the same library, things could get very confusing.

Proposal

1. So Ruben's initial catkin interface used the SKIP_PKG_CONFIG_GENERATION catkin_package() argument, and I think that should come back. If this is only needed for rosbuild compatibility, I think we can safely disable that. I removed it early on when I was debugging the setup, and planned on re-visiting it when we came to this discussion. @wjwwood, are the pkg-config files only really used by rosbuild?
2. Then, what we should do is set ${OROCOS_NO_AUTO_LINKING} to true and require that people use the CMake variables that are automatically defined by the orocos_use_package() calls just like normal CMake pattern. The issue here is that we want to avoid auto-linking all the time, so I propose we create a new macro called orocos_find_package() which is the same as calling orocos_use_package() with ${OROCOS_NO_AUTO_LINKING} set to true. We then use your patches which install the Orocos-generated pkg-config files, and Orocos will continue to use those.
3. Next, I think we shouid remove the "package.xml" parsing from UseOROCOS-RTT-helpers.cmake. I don't think this is being used internally, and before it was used in order to make things work in the rosbuild way, which now will just be confusing.
4. We then instruct people to not export the libraries generated by the orocos_*() macros via the catkin_project() macro, and for downstream packages to use orocos_find_package() to get the required variables.
5. Finally, all this should be documented clearly in the rtt_ros_integration documentation (:

Thoughts?

[meyerj]

Sounds good!

1. Then, what we should do is set ${OROCOS_NO_AUTO_LINKING} to true and require that people use the CMake variables that are automatically defined by the orocos_use_package() calls just like normal CMake pattern.

I did not like that auto-linking, too. orocos_find_package(pkg) should just define pkg_LIBRARIES, pkg_INCLUDE_DIRS, etc. variables. One option would be to automatically add all libraries, include directories to the OROCOS-RTT_LIBRARIES, OROCOS-RTT_INCLUDE_DIRS,... variables set defined in orocos-rtt-config.cmake (in analogy to catkin_* variables). That would probably result in a workflow like:

find_package(OROCOS-RTT REQUIRED)
include(${OROCOS-RTT_USE_FILE}) # also contains include_directories(), add_definitions(), ... (!)
orocos_find_package(another_package)

include_directories(${OROCOS-RTT_INCLUDE_DIRS})
orocos_component(my_component component.cpp)
target_link_libraries(my_component ${OROCOS-RTT_LIBRARIES})

1. Next, I think we shouid remove the "package.xml" parsing from UseOROCOS-RTT-helpers.cmake. I don't think this is being used internally, and before it was used in order to make things work in the rosbuild way, which now will just be confusing.

That would mean that all RTT dependencies have to be pulled in manually using orocos_find_package(), right? At least it would be possible to not enforce it, but to add another macro that calls orocos_find_package() for all dependencies recursively.

We then instruct people to not export the libraries generated by the orocos_*() macros via the catkin_project() macro, and for downstream packages to use orocos_find_package() to get the required variables.

This would be the logical consequence of having an own generate/find package mechanism for rtt. I do not see any sense for having both. Packages that only contain rtt stuff do not have to be catkin packages at all (just not call catkin_package() at all)?

The current pkg-config file generated by orocos_generate_package() is only valid in install-space and I think RTT has enforced that packages will be installed until now. Are there any plans to also enable usage of the orocos_generate_package/orocos_find_package in devel-space only? It should be straight-forward to generate a slightly different file in ${CATKIN_DEVEL_PREFIX}/lib/pkgconfig with no changes in orocos_use_package().

[smits]

On Wed, Aug 28, 2013 at 10:52 AM, Jonathan Bohren notifications@github.comwrote:

Yeah, this is something I've been putting off, mostly because I wanted to get feedback from @smits https://github.com/smits. Looks like now is a good time to start talking about it, at least.

I've been and am still being hammered with work for other projects, sorry for the ongoing delay.

So, to review the different buildsystems and the crud they do to the system: Catkin

• calling find_package(catkin COMPONENTS ...) will do two things for each package:
  1. it will execute any dependency's CFG_EXTRAS (CMake code)
  2. it will define a bunch of variables based on the arguments to each dependency's catkin_package() call
     • Calling the catkin_package()http://ros.org/doc/groovy/api/catkin/html/dev_guide/generated_cmake_api.html#catkin-packagemacro generates a CMake config file ( actionlibConfig.cmake https://gist.github.com/jbohren/6362488) and a pkg-config file (actionlib.pc https://gist.github.com/jbohren/6362499) for packages which depend on this package. Now, the pkg-config files appear to contain the same information as the CMake config files, and based on REP-0136https://github.com/ros-infrastructure/rep/blob/432d4ac866d5cbf1730f029dc06e035b9c2bafaa/rep-0136.rstit appears the only purpose of these pkg-config files is so that rosbuild packages can build against catkin-based packages.

Orocos

• Calling the orocos_component()/orocos_plugin()/etc macros constructs lists of linker args for each type of library
• Calling the orocos_generate_package()https://github.com/jhu-lcsr-forks/rtt/blob/66ce02896660f261a7f52644f64c05e0451ffbb9/UseOROCOS-RTT.cmake#L510macro generates a pkg-config file which lists all of the shared objects (libraries, typekits, plugins, components) as link libraries
• Calling the orocos_use_package(...)https://github.com/jhu-lcsr-forks/rtt/blob/d01316881cf9ae510c45370396612d55fd650468/UseOROCOS-RTT-helpers.cmake#L92macro sets variables similarly to the way Catkin does (i.e. normal find_package() behavior), but it will also automatically link all targets against all depedency libraries by default unless ${OROCOS_NO_AUTO_LINKING} is set to true.

However! When you include UseOROCOS-RTT.cmake, it automatically calls orocos_use_package() on every package manifest dependency (see herehttps://github.com/jhu-lcsr-forks/rtt/blob/66ce02896660f261a7f52644f64c05e0451ffbb9/UseOROCOS-RTT.cmake#L85). Previously, this read the manifest.xml, and currently, this will read the "package.xml" file in the ${CMAKE_CURRENT_SOURCE_DIR} and use xpath to get all tags in the package.xml file (see herehttps://github.com/jhu-lcsr-forks/rtt/blob/6605752/UseOROCOS-RTT-helpers.cmake#L73 ).

This last bit might actually be undesirable in this brave new catkin world, and if someone happens to both call orocos_generate_package()along with catkin_package() and exported the same library, things could get very confusing. Proposal

1.

So Ruben's initial catkin interface used the SKIP_PKG_CONFIG_GENERATION catkin_package() argument, and I think that should come back. If this is\ only needed for rosbuild compatibility, I think we can safely disable that. I removed it early on when I was debugging the setup, and planned on re-visiting it when we came to this discussion. @wjwwoodhttps://github.com/wjwwood, are the pkg-config files only really used by rosbuild? 2.

Then, what we should do is set ${OROCOS_NO_AUTO_LINKING} to true and require that people use the CMake variables that are automatically defined by the orocos_use_package() calls just like normal CMake pattern. The issue here is that we want to avoid auto-linking all the time, so I propose we create a new macro called orocos_find_package() which is the same as calling orocos_use_package() with ${OROCOS_NO_AUTO_LINKING} set to true. We then use your patches which install the Orocos-generated pkg-config files, and Orocos will continue to use those. 3.

Next, I think we shouid remove the "package.xml" parsing from UseOROCOS-RTT-helpers.cmake. I don't think this is being used internally, and before it was used in order to make things work in the rosbuild way, which now will just be confusing. 4.

We then instruct people to not export the libraries generated by the orocos_*() macros via the catkin_project() macro, and for downstream packages to use orocos_find_package() to get the required variables. 5.

Finally, all this should be documented clearly in the rtt_ros_integration documentation (:

Thoughts?

It looks ok, but we should have some simple use cases to test the behaviour and see if everything is covered. Overall we would like to have a cmake pattern that works both with and without catkin and l would not require any changes on the CMakeLists.txt side both for providing packages as for depending packages. I don't even know whether this is feasible.

Ruben

— Reply to this email directly or view it on GitHubhttps://github.com/jhu-lcsr-forks/rtt_ros_integration/issues/8#issuecomment-23399924 .

Ruben Smits, Phd Chief Technology Officer Intermodalics BVBA +32479511786 www.intermodalics.eu

[jbohren]

@meyerj

1. Then, what we should do is set ${OROCOS_NO_AUTO_LINKING} to true and require that people use the CMake variables that are automatically defined by the orocos_use_package() calls just like normal CMake pattern.

I did not like that auto-linking, too. orocos_find_package(pkg) should just define pkg_LIBRARIES, pkg_INCLUDE_DIRS, etc. variables. One option would be to automatically add all libraries, include directories to the OROCOS-RTT_LIBRARIES, OROCOS-RTT_INCLUDE_DIRS,... variables set defined in orocos-rtt-config.cmake (in analogy to catkin_* variables). That would probably result in a workflow like:

find_package(OROCOS-RTT REQUIRED)
include(${OROCOS-RTT_USE_FILE}) # also contains include_directories(), add_definitions(), ... (!)
orocos_find_package(another_package)

include_directories(${OROCOS-RTT_INCLUDE_DIRS})
orocos_component(my_component component.cpp)
target_link_libraries(my_component ${OROCOS-RTT_LIBRARIES})

Right, so right now, calling orocos_use_package(another_package) already creates those variables. The only issue is disabling auto-linking. So this new orocos_find_package(another_package) would be equivalent to calling orocos_use_package(another_package) with the auto-linking disabled. i.e:

Currently:

macro( orocos_use_package PACKAGE )
  # find package
  # define variables
  if (NOT OROCOS_NO_AUTO_LINKING)
    # do auto-linking
  endif()
endmacro()

Instead, we could do:

macro( orocos_find_package PACKAGE )
  # find package
  # define variables
endmacro()

macro( orocos_use_package PACKAGE )
  orocos_find_package(${PACKAGE})
  if (NOT OROCOS_NO_AUTO_LINKING)
    # do auto-linking
  endif()
endmacro()

This would remain backwards-compatible with the current Orocos cmake patterns.

@meyerj

1. Next, I think we shouid remove the "package.xml" parsing from UseOROCOS-RTT-helpers.cmake. I don't think this is being used internally, and before it was used in order to make things work in the rosbuild way, which now will just be confusing.

That would mean that all RTT dependencies have to be pulled in manually using orocos_find_package(), right? At least it would be possible to not enforce it, but to add another macro that calls orocos_find_package() for all dependencies recursively.

So I think, in line with the changes to remove ROS-specific junk from the Orocos/RTT core libraries, we could remove the automatic package.xml dependency parsing / autoloading from UseOROCOS-RTT-helpers.cmake. What we could do additionally is just move the package.xml parsing to a CMake macro defined in the rtt_ros package, and made available when you find rtt_ros. This macro could be called orocos_find_catkin_packages() or something. It feels strange naming it after "catkin" and then putting it in the rtt_ros package, but I think that's the right naming scheme.

@meyerj

We then instruct people to not export the libraries generated by the orocos_*() macros via the catkin_project() macro, and for downstream packages to use orocos_find_package() to get the required variables.

This would be the logical consequence of having an own generate/find package mechanism for rtt. I do not see any sense for having both. Packages that only contain rtt stuff do not have to be catkin packages at all (just not call catkin_package() at all)?

The problem with not calling catkin_package() is that then none of their build products would end up in the right devel space, among other problems.

The current pkg-config file generated by orocos_generate_package() is only valid in install-space and I think RTT has enforced that packages will be installed until now. Are there any plans to also enable usage of the orocos_generate_package/orocos_find_package in devel-space only? It should be straight-forward to generate a slightly different file in ${CATKIN_DEVEL_PREFIX}/lib/pkgconfig with no changes in orocos_use_package().

This wouldn't necessitate any changes to orocos_use_package(), but it would require changing where orocos_generate_package() defines the pkg-config prefix. Previously with rosbuild, these pkg-config files were "installed" to the package directory, so they could still be used. In this case, as you've illustrated, they are built to be used from ${CMAKE_INSTALL_PREFIX} which, when building with catkin, is the workspace install directory and not the devel directory.

We could remedy this by adding an orocos_generate_catkin_package() macro which would be a fork of orocos_generate_package() but creates the same files. Maybe the contents of the orocos_generate_package() macro could be made more modular to make more of it re-usable. Also note that this will also require a modified orocos_uninstall_target() macro.

@smits

It looks ok, but we should have some simple use cases to test the behaviour and see if everything is covered. Overall we would like to have a cmake pattern that works both with and without catkin and l would not require any changes on the CMakeLists.txt side both for providing packages as for depending packages. I don't even know whether this is feasible.

When you say with and without catkin, do you mean with both catkin and rosbuild, or with catkin and the orocos_* CMake macros?

[wjwwood]

@wjwwood, are the pkg-config files only really used by rosbuild?

Yes, I believe so. @dirk-thomas can confirm that.

[dirk-thomas]

Yes, the catkin packages only use the CMake config files. Dry packages and rospack are the only entities using the pkg-config files.

[jbohren]

@wjwwood @dirk-thomas cool, thanks for confirming that.

[meyerj]

Right, so right now, calling orocos_use_package(another_package) already creates those variables. The only issue is disabling auto-linking. So this new orocos_find_package(another_package) would be equivalent to calling orocos_use_package(another_package) with the auto-linking disabled. i.e:

+1

Next, I think we shouid remove the "package.xml" parsing from UseOROCOS-RTT-helpers.cmake. I don't think this is being used internally, and before it was used in order to make things work in the rosbuild way, which now will just be confusing.

That would mean that all RTT dependencies have to be pulled in manually using orocos_find_package(), right? At least it would be possible to not enforce it, but to add another macro that calls orocos_find_package() for all dependencies recursively.

So I think, in line with the changes to remove ROS-specific junk from the Orocos/RTT core libraries, we could remove the automatic package.xml dependency parsing / autoloading from UseOROCOS-RTT-helpers.cmake. What we could do additionally is just move the package.xml parsing to a CMake macro defined in the rtt_ros package, and made available when you find rtt_ros. This macro could be called orocos_find_catkin_packages() or something. It feels strange naming it after "catkin" and then putting it in the rtt_ros package, but I think that's the right naming scheme.

+1

I also worked on a variant that supports rosbuild packages:

• https://github.com/meyerj/rtt/tree/rosbuild_support
• https://github.com/meyerj/rtt_ros_integration/tree/rosbuild_support

The changes to your versions are only minimal and my old rosbuild projects with some inter-dependencies compile successfully with these branches (There are still runtime issues with the missing ROS support in the RTT ComponentLoader).

In the rosbuild_support branch I added the rosbuild support to UseOROCOS-RTT-helpers.cmake if ROSBUILD_init_called is set. But adding this to rtt_ros and removing ROS from rtt core is probably the cleaner solution.

The current pkg-config file generated by orocos_generate_package() is only valid in install-space and I think RTT has enforced that packages will be installed until now. Are there any plans to also enable usage of the orocos_generate_package/orocos_find_package in devel-space only? It should be straight-forward to generate a slightly different file in ${CATKIN_DEVEL_PREFIX}/lib/pkgconfig with no changes in orocos_use_package().

This wouldn't necessitate any changes to orocos_use_package(), but it would require changing where orocos_generate_package() defines the pkg-config prefix. Previously with rosbuild, these pkg-config files were "installed" to the package directory, so they could still be used. In this case, as you've illustrated, they are built to be used from ${CMAKE_INSTALL_PREFIX} which, when building with catkin, is the workspace install directory and not the devel directory.

We could remedy this by adding an orocos_generate_catkin_package() macro which would be a fork of orocos_generate_package() but creates the same files. Maybe the contents of the orocos_generate_package() macro could be made more modular to make more of it re-usable. Also note that this will also require a modified orocos_uninstall_target() macro.

I think we do not need a new macro. It's not a problem to create two different .pc files, one for the devel-space and another that is installed, with different prefixes. I already have done that for rosbuild packages here.

The only issue is that orocos_generate_package() creates pkg-config files with the -${OROCOS_TARGET} suffix while catkin/rospack uses the version without suffix. I also did some experiments with that in my rosbuild_support branch. I just had to swap the order of package search names in https://github.com/meyerj/rtt/blob/rosbuild_support/UseOROCOS-RTT-helpers.cmake#L142. orocos_use_package() relies on the files with suffix then, and catkin/rosbuild use the version generated by catkin (without orocos libraries). I think that would be fine? Libs/include dirs declared in catkin are pulled in by catkin/rosbuild. Orocos-specific libs have to be pulled in using the orocos_find_package()/orocos_use_package() macros. Two different .pc files (or more if there are more targets available).

I would like to keep rosbuild support if possible. The differences in the required cmake code and the rtt_ros service are not so big and it would be convenient for existing projects. It would be okay to require some changes in the manifests and in cmake, but I don't see a good reason to drop rosbuild support totally.

[jbohren]

@meyerj

The current pkg-config file generated by orocos_generate_package() is only valid in install-space and I think RTT has enforced that packages will be installed until now. Are there any plans to also enable usage of the orocos_generate_package/orocos_find_package in devel-space only? It should be straight-forward to generate a slightly different file in ${CATKIN_DEVEL_PREFIX}/lib/pkgconfig with no changes in orocos_use_package().

This wouldn't necessitate any changes to orocos_use_package(), but it would require changing where orocos_generate_package() defines the pkg-config prefix. Previously with rosbuild, these pkg-config files were "installed" to the package directory, so they could still be used. In this case, as you've illustrated, they are built to be used from ${CMAKE_INSTALL_PREFIX} which, when building with catkin, is the workspace install directory and not the devel directory.

We could remedy this by adding an orocos_generate_catkin_package() macro which would be a fork of orocos_generate_package() but creates the same files. Maybe the contents of the orocos_generate_package() macro could be made more modular to make more of it re-usable. Also note that this will also require a modified orocos_uninstall_target() macro.

I think we do not need a new macro. It's not a problem to create two different .pc files, one for the devel-space and another that is installed, with different prefixes. I already have done that for rosbuild packages here.

Right, we can do this, but part of the goal of all of this is to remove all of the catkin/ros-specific stuff from the Orocos upstream, aside from lightweight package.xml files. If we added the ${CATKIN_DEVEL_PREFIX} variable to the normal orocos_generate_package() macro, then that would involve putting catkin code into the Orocos upstream.

The only issue is that orocos_generate_package() creates pkg-config files with the -${OROCOS_TARGET} suffix while catkin/rospack uses the version without suffix. I also did some experiments with that in my rosbuild_support branch. I just had to swap the order of package search names in https://github.com/meyerj/rtt/blob/rosbuild_support/UseOROCOS-RTT-helpers.cmake#L142. orocos_use_package() relies on the files with suffix then, and catkin/rosbuild use the version generated by catkin (without orocos libraries). I think that would be fine? Libs/include dirs declared in catkin are pulled in by catkin/rosbuild. Orocos-specific libs have to be pulled in using the orocos_find_package()/orocos_use_package() macros. Two different .pc files (or more if there are more targets available).

Yeah! This sounds like it should work well to support both kinds of pkg-config files. That's also nice because it removes the need for people to override the normal catkin pkg-config generation behavior.

I would like to keep rosbuild support if possible. The differences in the required cmake code and the rtt_ros service are not so big and it would be convenient for existing projects. It would be okay to require some changes in the manifests and in cmake, but I don't see a good reason to drop rosbuild support totally.

To be clear, you mean support for rosbuild packages to depend on catkin-based orocos packages, right?

[meyerj]

I created a separate thread #9 to discuss the rosbuild support, as it is off-topic here.

To summarize, there are currently the following proposals in this thread concerning pkg-config:

1. Add a orocos_find_package() macro that only sets variables without auto-linking. orocos_use_package() calls orocos_find_package() internally and does the auto-linking if OROCOS_NO_AUTO_LINKING is not set.
2. The order of ${PACKAGE} ${PACKAGE}-${OROCOS_TARGET} search names in the pkg_search_module() call should be swapped to prefer the files generated by orocos_generate_package() over the one generated by catkin.
3. Parsing manifest.xml/package.xml or crawling packages with rospack will be removed from UseOROCOS-RTT.cmake. Instead, a new macro is added to rtt_ros. I would prefer orocos_find_ros_package() over orocos_find_catkin_package().

Regarding 3., I am not sure if the manifest.xml files only have been used in ROS packages. I remember that they have been advertised as a general mechanism to pull in dependent packages in pure Orocos (see e.g. http://www.orocos.org/forum/orocos/orocos-users/orocos-toolchain-23-now-require-manifestxml-file). It's just "by accident" that they use the same file name as ROS. If this is the case, this part should remain in rtt and we do not need an additional macro in rtt_ros. Perhaps @smits can again comment on this?

[psoetens]

I created a separate thread #9 to discuss the rosbuild support, as it is off-topic here.

To summarize, there are currently the following proposals in this thread concerning pkg-config:

Add a orocos_find_package() macro that only sets variables without auto-linking. orocos_use_package() calls orocos_find_package() internally and does the auto-linking if OROCOS_NO_AUTO_LINKING is not set.

OK

The order of ${PACKAGE} ${PACKAGE}-${OROCOS_TARGET} search names in the pkg_search_module() call should be swapped to prefer the files generated by orocos_generate_package() over the one generated by catkin.

OK

Parsing manifest.xml/package.xml or crawling packages with rospack will be removed from UseOROCOS-RTT.cmake. Instead, a new macro is added to rtt_ros. I would prefer orocos_find_ros_package() over orocos_find_catkin_package(). Regarding 3., I am not sure if the manifest.xml files only have been used in ROS packages. I remember that they have been advertised as a general mechanism to pull in dependent packages in pure Orocos (see e.g. http://www.orocos.org/forum/orocos/orocos-users/orocos-toolchain-23-now-require-manifestxml-file). It's just "by accident" that they use the same file name as ROS. If this is the case, this part should remain in rtt and we do not need an additional macro in rtt_ros. Perhaps @smits can again comment on this?

Correct, the manifest.xml file is used by Autoproj, the native Orocos meta build system. We hoped to keep packages compatible with both Autoproj and Catkin (autoproj and rosbuild were compatible wrt manifest.xml), but the swing to package.xml and the redesign of its contents make this a difficult split-position. Autoproj will ignore the package.xml file, so we can keep manifest.xml... if catkin ignores it ?

In order to keep auto-linking packages working, we'll have to keep manifest.xml and this crawling. If proper macro's are provided, I think we can start deprecating auto-linking in Orocos as well, but crawling to set the PKG_CONFIG_PATHS I would keep.

[jbohren]

@psoetens

Regarding 3., I am not sure if the manifest.xml files only have been used in ROS packages. I remember that they have been advertised as a general mechanism to pull in dependent packages in pure Orocos (see e.g. http://www.orocos.org/forum/orocos/orocos-users/orocos-toolchain-23-now-require-manifestxml-file). It's just "by accident" that they use the same file name as ROS. If this is the case, this part should remain in rtt and we do not need an additional macro in rtt_ros. Perhaps @smits can again comment on this?

Correct, the manifest.xml file is used by Autoproj, the native Orocos meta build system. We hoped to keep packages compatible with both Autoproj and Catkin (autoproj and rosbuild were compatible wrt manifest.xml), but the swing to package.xml and the redesign of its contents make this a difficult split-position. Autoproj will ignore the package.xml file, so we can keep manifest.xml... if catkin ignores it ?

Yeah catkin will ignore the manifest.xml files.

In order to keep auto-linking packages working, we'll have to keep manifest.xml and this crawling. If proper macro's are provided, I think we can start deprecating auto-linking in Orocos as well, but crawling to set the PKG_CONFIG_PATHS I would keep.

Ok, I'll change that back to manifest.xml in the orocos_get_manifest_deps() CMake macro. I'll also make a point in the documentation about which manifest.xml files it's talking about.

Is there a specification somewhere for the Orocos manifest.xml files?

[jbohren]

There are a lot of topics in this issue, some of them have to do with rtt_ros_integration, and others require changes in rtt, itself.

Initial Ticket

The initial problem was that both Catkin's catkin_package() and Orocos's orocos_generate_package() generate pkg-config files. Catkin uses pkg-config files to support rosbuild packages depending on catkin packages, and orocos uses pkg-config files to associate package names with compile/linker flags and their dependencies. Catkin names these "PACKAGE_NAME.pc" and Orocos names them "PACKAGE_NAME-OROCOS_TARGET.pc" but when Orocos goes to retrieve pkg-config files in orocos_use_package(), it looked for both names.

This first necessitated a change in RTT to switch the search order of the pkg-config files: https://github.com/jhu-lcsr-forks/rtt/commit/8d465b341005238a9dd6fbd03632d7619aca148e#commitcomment-3978504

Then, the other issue was that Orocos pkg-config files were not put into the Catkin devel prefix, so they were only available once installed. The decision was to make the Orocos UseOROCOS-RTT.cmake detect if the client is either rosbuild or Catkin, and determine where to place additional pkg-config files, if necessary. Since these changes also involved reviving the rosbuild support, its implementation is covered in several commits:

https://github.com/jhu-lcsr-forks/rtt/compare/d607a06...3c47308?w=1

Other changes to RTT

There were other Catkin-supporting features discussed and added, which will be detailed further here: https://github.com/jhu-lcsr-forks/rtt/issues/4

[jbohren]

Also, the orocos pkg-config file which is now generated in a catkin develspace looks like this (as of jhu-lcsr-forks/rtt@6ac1a42 ):

# Orocos pkg-config file generated by orocos_generate_package() 
# This pkg-config file is for use in a Catkin devel space
prefix=/home/jbohren/ws/groovy/underlay/devel
libdir=${prefix}/lib
includedir=${prefix}/include/orocos
orocos_libdir=${libdir}/orocos/gnulinux/rtt_rosnode

Name: rtt_rosnode-gnulinux
Description: rtt_rosnode-gnulinux package for Orocos
Requires: orocos-rtt-gnulinux 
Version: 1.0
Libs:  -L${orocos_libdir}/plugins  -lrtt_rosnode-gnulinux
Cflags: -I${includedir} 

[meyerj]

I am not sure if this catkin pkg-config file is sufficient to orocos_use_package(rtt_rosnode) from devel-space without involving catkin. The devel-space only contains the binaries and some files generated by catkin.

Not sure without doing some tests, but includedir should (additionally) point to the source directory here or be omitted totally.

[jbohren]

I am not sure if this catkin pkg-config file is sufficient to orocos_use_package(rtt_rosnode) from devel-space without involving catkin. The devel-space only contains the binaries and some files generated by catkin.

Yeah, this might need some tweaking, but that's why we need to set up some scripts to generate different build environments to fully test it. Before I wasn't using these pkg-config files at all, so none of my projects really test that out until I change them to do that. In the meantime, none of these changes have broken stuff on my system.

[eacousineau]

@smits Can I ask why there is a desire to keep catkin separate? As a ROS user, catkin seems like a versatile build system that it itself is independent of ROS / interfaces well with system dependencies.