Package version comparisons generated by the debian generator are broken

[meyerj]

The debian generator of bloom translates version dependencies found in package.xml files to version restrictions in the Debian control file, without any interpretation of the version string: generators/debian/generator.py:197.

While this approach works fine in the common case of depending on a minimum version of a ROS package like in

<depend version_gte="1.12.12">roscpp</depend>

which is translated to

Depends: ros-kinetic-roscpp (>= 1.12.12)

for the control file, it is basically broken for version_lte, version_eq or version_gt because only very few Debian or Ubuntu packages have version numbers that exactly follow the major.minor.patch version scheme. Version specification and comparison on Debian packages is more complex, see https://www.debian.org/doc/debian-policy/#s-f-version for reference.

Examples that would fail or behave unexpectedly:

• Equality:

  <depend version_eq="1.12.12">roscpp</depend>
  Depends: ros-kinetic-roscpp (= 1.12.12)

  is not satisfied by ros-kinetic-roscpp=1.12.12-0xenial-20171116-193322-0800 because it's not equal to 1.12.12:

  dpkg: dependency problems prevent configuration of foo:
  foo depends on ros-kinetic-roscpp (= 1.12.12); however:
  Version of ros-kinetic-roscpp on system is 1.12.12-0xenial-20171116-193322-0800.

• Earlier or equal:

  <depend version_lte="1.12.12">roscpp</depend>
  Depends: ros-kinetic-roscpp (<= 1.12.12)

  would be satisfied by ros-kinetic-roscpp=1.12.7-0xenial-20170613-165746-0800, but but not by ros-kinetic-roscpp=1.12.12-0xenial-20171116-193322-0800 because it's greater than 1.12.12:

  dpkg: dependency problems prevent configuration of foo:
  foo depends on ros-kinetic-roscpp (<= 1.12.12); however:
  Version of ros-kinetic-roscpp on system is 1.12.12-0xenial-20171116-193322-0800.

• Later:

  <depend version_gt="1.12.12">roscpp</depend>
  Depends: ros-kinetic-roscpp (>> 1.12.12)

  would unexpectedly be satisfied by ros-kinetic-roscpp=1.12.12-0xenial-20171116-193322-0800 because it's greater than 1.12.12:

  $ dpkg-query -s foo ros-kinetic-roscpp
  Package: foo
  Status: install ok installed
  Priority: optional
  Section: misc
  Installed-Size: 9
  Maintainer: Johannes Meyer <johannes@intermodalics.eu>
  Architecture: all
  Multi-Arch: foreign
  Version: 1.0
  Depends: ros-kinetic-roscpp (>> 1.12.12)
  Description: Dummy package to fulfill package dependencies
  This is a dummy package that makes Debian's package management
  system believe that equivalents to packages on which other
  packages depend on are actually installed.  Deinstallation of
  this package is only possible when all pending dependency issues
  are properly resolved in some other way.
  .
  Please note that this is a crude hack and if thoughtlessly used
  might possibly do damage to your packaging system.  And please
  note as well that using it is not the recommended way of dealing
  with broken dependencies.  It is better to file a bug report.
  
  Package: ros-kinetic-roscpp
  Status: install ok installed
  Priority: extra
  Section: misc
  Installed-Size: 2420
  Maintainer: Dirk Thomas <dthomas@osrfoundation.org>
  Architecture: amd64
  Version: 1.12.12-0xenial-20171116-193322-0800
  Depends: libboost-filesystem1.58.0, libboost-system1.58.0, libboost-thread1.58.0, libc6 (>= 2.14), libgcc1 (>= 1:3.0), libstdc++6 (>= 5.2), ros-kinetic-cpp-common (>= 0.3.17), ros-kinetic-message-runtime, ros-kinetic-rosconsole, ros-kinetic-roscpp-serialization, ros-kinetic-roscpp-traits (>= 0.3.17), ros-kinetic-rosgraph-msgs (>= 1.10.3), ros-kinetic-rostime (>= 0.6.4), ros-kinetic-std-msgs, ros-kinetic-xmlrpcpp
  Description: roscpp is a C++ implementation of ROS.
  It provides a client library that enables C++ programmers to quickly interface with ROS Topics, Services, and Parameters. roscpp is the most widely used ROS client library and is designed to be the high-performance library for ROS.
  Homepage: http://ros.org/wiki/roscpp

• Later, with or without equal, in combination with non-zero epoch numbers:

  <depend version_gte="9.9.9">zlib</depend>
  Depends: zlib1g-dev (>= 9.9.9)

  would be satisfied by zlib1g-dev=1:1.2.8.dfsg-2ubuntu4.1 because the epoch number before the colon is greater than zero, the default epoch assumed if none is specified like for 9.9.9:

  1:1.2.8.dfsg-2ubuntu4.1
  $ dpkg-query -s foo zlib1g-dev
  Package: foo
  Status: install ok installed
  Priority: optional
  Section: misc
  Installed-Size: 9
  Maintainer: Johannes Meyer <johannes@intermodalics.eu>
  Architecture: all
  Multi-Arch: foreign
  Version: 1.0
  Depends: zlib1g-dev (>= 9.9.9)
  Description: Dummy package to fulfill package dependencies
  This is a dummy package that makes Debian's package management
  system believe that equivalents to packages on which other
  packages depend on are actually installed.  Deinstallation of
  this package is only possible when all pending dependency issues
  are properly resolved in some other way.
  .
  Please note that this is a crude hack and if thoughtlessly used
  might possibly do damage to your packaging system.  And please
  note as well that using it is not the recommended way of dealing
  with broken dependencies.  It is better to file a bug report.
  
  Package: zlib1g-dev
  Status: install ok installed
  Priority: optional
  Section: libdevel
  Installed-Size: 415
  Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
  Architecture: amd64
  Multi-Arch: same
  Source: zlib
  Version: 1:1.2.8.dfsg-2ubuntu4.1
  Provides: libz-dev
  Depends: zlib1g (= 1:1.2.8.dfsg-2ubuntu4.1), libc6-dev | libc-dev
  Conflicts: zlib1-dev
  Description: compression library - development
  zlib is a library implementing the deflate compression method found
  in gzip and PKZIP.  This package includes the development support
  files.
  Homepage: http://zlib.net/
  Original-Maintainer: Mark Brown <broonie@debian.org>

(The dummy package foo has been generated with equivs to emulate a control file generated by bloom.)

In the version_gte or version_lt case the generated comparison probably behaves as expected for most packages, but only in case the respective Debian version does have a zero or unspecified epoch.

Am I missing something here? Or is it allowed to specify exact Debian version strings in package.xml? The latter would break platform-independence and probably some other tools.

I am not sure what would be the best way to fix this, but maybe bloom should at least emit a warning if a package specifies version_eq or version_lte dependencies because they most likely cannot be satisfied and the generated debian package cannot be installed.

[wjwwood]

Am I missing something here?

No I think we just have never used anything other than version_gte in practice. It sounds practically broken, but it's not something we can easily fix in bloom I think, and is just a side-effect of how we do versions in our debs.

Or is it allowed to specify exact Debian version strings in package.xml?

No the idea is that you do not do that, but maybe @dirk-thomas or @tfoote disagree with me there.

I am not sure what would be the best way to fix this, but maybe bloom should at least emit a warning if a package specifies version_eq or version_lte dependencies because they most likely cannot be satisfied and the generated debian package cannot be installed.

That's probably a good idea. I don't have time to do it, but if you or someone else has time to make a pr for that I think we'd consider it, at the very least until we can come up with a fix.

[tfoote]

Yeah, gte is the only one I know of using in practice to enforce that a new required feature has been made available.

Or is it allowed to specify exact Debian version strings in package.xml?

No the idea is that you do not do that, but maybe @dirk-thomas or @tfoote disagree with me there.

The package.xml needs to be platform independent. We generate into rpm, arch, openembedded etc.

The goal of these statements is to do the package version comparisions on the release versions only ignoring the build number since the package cannot know about the build number. Is there a way we can tell debian to limit the comparison to just the package version?

[nuclearsandwich]

I forgot that I'd asked @j-rivero for suggestions here to see if there was a debian version comparator that would serve us but there unfortunately isn't one.

One possible way to handle version_eq would be to expand one package.xml version constraint to two debian version constraints. For example

<depend version_eq="X.Y.Z">roscpp</depend>

could be made to expand to

Depends: ros-kinetic-roscpp (>= X.Y.Z) ros-kinetic-roscpp (<< X.Y.Z+1)

I don't have a suggestion for the lte case but I agree that printing a warning of possible consequences when these conditions are used is a good idea.

[tfoote]

Couldn't the following mappings be made to work where +1 -1 is actually mutating the number in the debian generator. lte: << X.Y.Z+1 lt: << X.Y.Z gt: >=X.Y.Z+1 gte: >=X.Y.Z eq: >=X.Y.Z && << X.Y.Z+1

This will assume that the smallet increment is a patch release. But I think that's the basic philosophy of this platform agnostic approach.

[meyerj]

I assume that what @tfoote proposed would work and was thinking about the same.

Version comparisons for non-ROS packages that have a non-zero epoch are still broken (see zlib case above), but I guess there is no simple solution and tools like dpkg-shlibdeps will already add the correct dependencies including epoch if necessary.