Binary deb message packages include many unecessary runtime dependencies

[emersonknapp]

Bug report

This is to start a conversation about this hopefully - not sure exactly what the solution is right now.

I'm trying to create a "relatively minimal" installation of just a few ROS packages for use in a specific context where size/bandwidth is at a premium.

I start with the ubuntu:jammy base Docker image, which is ~78MB.

I run the following script to get the ROS apt repositories available:

set -euxo pipefail

LANG=en_US.UTF-8
DEBIAN_FRONTEND=noninteractive
apt-get update

echo 'Etc/UTC' > /etc/timezone
apt-get update
apt-get install --no-install-recommends --quiet --yes \
    ca-certificates curl gnupg2 locales

update-ca-certificates
curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu jammy main" |\
  tee /etc/apt/sources.list.d/ros2.list > /dev/null

rm -rf /var/lib/apt/lists/*
apt-get clean

Which adds a little overhead, and bumps the size to 107MB.

Now, in that container, I go to apt install ros-iron-builtin-interfaces which gives

0 upgraded, 205 newly installed, 0 to remove and 3 not upgraded.
Need to get 114 MB of archives.
After this operation, 431 MB of additional disk space will be used.

If I add --no-install-recommends, that is mitigated quite a bit to

0 upgraded, 126 newly installed, 0 to remove and 3 not upgraded.
Need to get 48.5 MB of archives.
After this operation, 203 MB of additional disk space will be used.

But that still seems like quite a lot. The full list of packages (with no-recommends) is as follows:

  cmake cmake-data dh-elpa-helper docutils-common emacsen-common google-mock googletest libarchive13 libatomic1 libblas3 libc-dev-bin libc6-dev libcrypt-dev libexpat1 libexpat1-dev libgfortran5 libgtest-dev libicu70 libjs-jquery libjs-sphinxdoc
  libjs-underscore libjsoncpp25 liblapack3 libmpdec3 libnsl-dev libpython3-dev libpython3-stdlib libpython3.10 libpython3.10-dev libpython3.10-minimal libpython3.10-stdlib libquadmath0 librhash0 libtirpc-dev libuv1 libxml2 linux-libc-dev media-types
  python3 python3-argcomplete python3-attr python3-catkin-pkg-modules python3-dateutil python3-dev python3-distutils python3-docutils python3-empy python3-importlib-metadata python3-iniconfig python3-lark python3-lib2to3 python3-minimal
  python3-more-itertools python3-numpy python3-packaging python3-pkg-resources python3-pluggy python3-py python3-pyparsing python3-pytest python3-roman python3-setuptools python3-six python3-toml python3-zipp python3.10 python3.10-dev
  python3.10-minimal ros-iron-ament-cmake ros-iron-ament-cmake-core ros-iron-ament-cmake-export-definitions ros-iron-ament-cmake-export-dependencies ros-iron-ament-cmake-export-include-directories ros-iron-ament-cmake-export-interfaces
  ros-iron-ament-cmake-export-libraries ros-iron-ament-cmake-export-link-flags ros-iron-ament-cmake-export-targets ros-iron-ament-cmake-gen-version-h ros-iron-ament-cmake-gmock ros-iron-ament-cmake-gtest ros-iron-ament-cmake-include-directories
  ros-iron-ament-cmake-libraries ros-iron-ament-cmake-pytest ros-iron-ament-cmake-python ros-iron-ament-cmake-ros ros-iron-ament-cmake-target-dependencies ros-iron-ament-cmake-test ros-iron-ament-cmake-version ros-iron-ament-index-python
  ros-iron-ament-package ros-iron-builtin-interfaces ros-iron-domain-coordinator ros-iron-fastcdr ros-iron-fastrtps-cmake-module ros-iron-gmock-vendor ros-iron-gtest-vendor ros-iron-python-cmake-module ros-iron-rcpputils ros-iron-rcutils ros-iron-rmw
  ros-iron-ros-workspace ros-iron-rosidl-adapter ros-iron-rosidl-cli ros-iron-rosidl-core-runtime ros-iron-rosidl-dynamic-typesupport ros-iron-rosidl-generator-c ros-iron-rosidl-generator-cpp ros-iron-rosidl-generator-py
  ros-iron-rosidl-generator-type-description ros-iron-rosidl-parser ros-iron-rosidl-pycommon ros-iron-rosidl-runtime-c ros-iron-rosidl-runtime-cpp ros-iron-rosidl-typesupport-c ros-iron-rosidl-typesupport-cpp ros-iron-rosidl-typesupport-fastrtps-c
  ros-iron-rosidl-typesupport-fastrtps-cpp ros-iron-rosidl-typesupport-interface ros-iron-rosidl-typesupport-introspection-c ros-iron-rosidl-typesupport-introspection-cpp ros-iron-rpyutils rpcsvc-proto sgml-base tzdata xml-core zlib1g-dev

Some key names in here that strike me as clearly not required for runtime use for this package (not a complete list):

• cmake
• google-mock
• googletest
• libjs-sphinxdoc
• python3-numpy
• ros-iron-ament-*
• ros-iron-rosidl-adapter
• ros-iron-rosidl-parser
• ros-iron-rosidl-* (excepting runtime and typesupport, those are probably necessary)

So, I'm wondering how we could start approaching making these much less heavyweight by stripping all the build-time dependencies.

For a comparison of some other package providing some meaningful unit of functionality, ros-iron-rosbag2-storage-mcap only will pull in 6kB to this bare environment. Admittedly they're doing very different things, but I would expect the amount of generated code and dynamically loaded dependencies for a message package to be quite small too!

0 upgraded, 21 newly installed, 0 to remove and 3 not upgraded.
Need to get 1479 kB of archives.
After this operation, 6026 kB of additional disk space will be used.

Tangent: An additional question would be if it'd be possible to install "just the C++" for the messages, since the Python may not be used in a particular environment and brings pretty heavy dependencies.

[emersonknapp]

Here's the dotfile for the apt dependency tree for rosidl-core-runtime where it seems like most of the tree is coming from ros-iron-rosidl-core-runtime.zip

[sloretz]

IIRC correctly from the meeting this was discussed in, this is caused by packages not being separated into -dev variants. That is every ROS package is assumed to have both stuff needed to build against it and run with it. Some people have custom scripts to do this separation: https://github.com/jspricke/ros-deb-builder-action 🧇