search

osrf / docker_images

A repository to hold definitions of docker images maintained by OSRF
Apache License 2.0
569 stars 170 forks source link

[Feature Request] Images mixing ROS + Gazebo #412

Closed moriarty closed 4 years ago

moriarty commented 4 years ago

@ruffsl & @mikaelarguedas I'd like to be able to simply mix two overlapping LTS versions of ROS and Gazebo if it's unofficially supported by getting ROS from https://package.ros.org and Gazebo from https://packages.osrfoundation.org ... for some reason I find myself doing this a lot.

First: I know osrf/ros:melodic-desktop-full does have a version of gazebo, but it's really old.

Similar motivation as this issue https://github.com/ros-simulation/gazebo_ros_pkgs/issues/1088 @chapulina & @j-rivero - but to take advantage of those newly released packages via a ready to go Docker base layer to build on top of. I think this would help get ROS users transitioning to newer versions of Gazebo faster.

For example, these new tags would be created:

FROM osrf/ros:melodic-gazebo9
FROM osrf/ros:melodic-gazebo11
FROM osrf/ros:melodic-ignition3 (?)
FROM osrf/ros2:dashing-gazebo11

For the easiest case, osrf/ros:melodic-gazebo9 this is simply osrf/ros:melodic-desktop-full but with the latest gazebo 9 (as of today that is 9.13.1) with ros melodic instead of gazebo 9.0.0 which comes with melodic-desktop-full

Suggestion for osrf/ros:melodic-gazebo9

FROM osrf/ros:melodic-desktop-full

RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys D2486D2DD83DB69272AFE98867170598AF249743 \
 && . /etc/os-release \
 && echo "deb http://packages.osrfoundation.org/gazebo/$ID-stable `lsb_release -sc` main" > /etc/apt/sources.list.d/gazebo-latest.list

RUN apt-get update \
 && apt-get upgrade -y

For other cases, I would assume a similar process, start with osrf/ros:{ros_version}-desktop-full add the apt source: https://packages.osrfoundation.org and then install a never version of ros-{ros_version}-gazebo{gazebo_version}-ros-pkgs and let apt do it's best to remove the older stuff and install the newer stuff which requires an update, upgrade and then install.

Suggestion for osrf/ros:melodic-gazebo11

FROM osrf/ros:melodic-desktop-full

RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys D2486D2DD83DB69272AFE98867170598AF249743 \
 && . /etc/os-release \
 && echo "deb http://packages.osrfoundation.org/gazebo/$ID-stable `lsb_release -sc` main" > /etc/apt/sources.list.d/gazebo-latest.list

RUN apt-get update \
 && apt-get upgrade -y \
 && apt-get install -y -q --no-install-recommends \
  ros-melodic-gazebo11-ros-pkgs
mikaelarguedas commented 4 years ago

In general ROS relies on the fact that it declares which version of dependencies it targets (REP3 and REP2000) and the logic for installing is "we should use what is provided upstream and not play dependency whack-a-mole".

First: I know osrf/ros:melodic-desktop-full does have a version of gazebo, but it's really old

Indeed the ROS images leverage the upstream ubuntu repositories for all its dependencies and this means that the version of gazebo used is the one in ubuntu (in most cases) that doesnt get updated easily. There are 2 kinds of "really old":

For the first one, I think it's a reasonable use-case for users. ROS specifies which version of gazebo it targets but upgrading to a more recent ABI compatible version should be supported and straight forward. Two questions that come to mind:

For the second case. I think it's well summarized in your statement "and let apt do it's best to remove the older stuff and install the newer stuff which requires an update, upgrade and then install." This is a non-supported combination of ROS and Gazebo and doing this ends up removing a potentially not known set of packages and may leave you with a broken configuration. Any ROS package released with a dependency on Gazebo will either not be installable or be installed but built against an ABI incompatible version of Gazebo. As a result I dont think this would belong in the official images.

ruffsl commented 4 years ago

I agree with @mikaelarguedas , as for mixing ROS and Gazebo versions isn't something that I think necessitates a dedicated image here. I think you'd be better served by writing a derived Dockerfile for a custom child image that simply builds FROM ROS repo and installs the gazebo debian.

For the second case. I think it's well summarized in your statement "and let apt do it's best to remove the older stuff and install the newer stuff which requires an update, upgrade and then install."

It should be noted that while starting from ros-desktop-full may save you from having to install more desktop related packages, removing and installing a different gazebo packages would strictly increase the images size to something larger that installing desktop+gazebo from say ros-base tag given how file deltas in image layers work.

moriarty commented 4 years ago

Sorry I'm replying out of order

I think you'd be better served by writing a derived Dockerfile for a custom child image that simply builds FROM ROS repo and installs the gazebo debian.

Yes I didn't specify it but I would not expect production ready images. In production environments I'd do something completely different and not base FROM on any the official images, probably avoid the debians all together to enable fine grain pinning of packages which is assuming a decent amount of test infrastructure is in place which is how I assume commercial users are doing things already...

This feature request was based on my experience from my free time projects and past experience supporting researchers on the Fetch Robots. I frequently find myself writing dockers with a new or old version of Gazebo + ROS for some reason or another. Especially during transitions between LTS releases.

Sorry about the "really old" comment. Yes I know 9.0.0 v 9.13.1 seems not that old, but IMHO there are a lot of small annoying issues with 9.0.0 like the api.ignitionfuel.org -> fuel.ignitionrobotics.org change which prevents it a client from starting without adding a fix to ~/.ignition/fuel/config.yaml

why do that for Gazebo and not other dependencies (OpenCV, PCL, Ignition...) ?

Re: OpenCV, PCL - in commercial settings, I'm assuming everyone's already upgrading or downgrading and version pinning their OpenCV and PCL probably compiling them with magic optimization settings specific to hardware, also they're not under the osrf foundation so they would be far out of scope.

Gazebo + ROS, and soon Ignition (https://www.ros.org/reps/rep-2000.html#foxy-fitzroy-may-2020-may-2023) are on the other hand all osrf projects and wouldn't think an argument could be made that it's not that far out of scope. In fact, the biggest use case I imagine would be to accelerate testing and version upgrading. To limit the scope even further, I wouldn't suggest every version of ROS / Gazebo or even every ROS LTS with overlapping Gazebo LTS, but simply the few that debian packages of ros_gazebo_packages are sort-of supported and to some extent tested

If a user has some packages running on Melodic, they're going to jump to Noetic, with this Feature Request they could quickly do some testing that their existing packages perform as expected with: Ubuntu 18.04 + ROS Melodic + Gazebo 11 before making the more extreme jump to 20.04, Noetic, Gazebo 11 - which will bring with it all the other differences as defined in https://www.ros.org/reps/rep-0003.html#noetic-ninjemys-may-2020-may-2025

Especially if the images aren't under the official gazebo or ros dockerhub listings, but under the u/osrf/ images There is already a note here: https://hub.docker.com/_/gazebo

The OSRF's Docker Hub organization profile contains a Gazebo repo at osrf/gazebo which is based on this repo but includes additional tags for these advanced use cases.

Foxy's going to be really interesting because- assuming most users are jumping form LTS -> LTS, there will be users on 18.04 + Dashing + Gazebo 9 who will be jumping to 20.04 + Foxy + { Gazebo 11 or Ignition Citadel } where they might want to test Gazebo11 vs Ignition Citadel quickly before they make the jump.

It should be noted that while starting from ros-desktop-full may save you from having to install more desktop related packages, removing and installing a different gazebo packages would strictly increase the images size to something larger that installing desktop+gazebo from say ros-base tag given how file deltas in image layers work.

Yes, again the cleanest way for image size would be to start with new base FROM ubuntu and setup the osrf and ros package repositories and then install ros-${ROS_DISTRO}-gazebo${GAZEBO_VERSION}-pkgs

example only still missing at least build-essential etc.

FROM ubuntu:18.04

RUN apt-get update \
 && apt-get install -q -y --no-install-recommends \
    gnupg2 \
    lsb-release

RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654 \
 && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys D2486D2DD83DB69272AFE98867170598AF249743 \
 && . /etc/os-release \
 && echo "deb http://packages.osrfoundation.org/gazebo/$ID-stable `lsb_release -sc` main" > /etc/apt/sources.list.d/gazebo-latest.list \
 && echo "deb http://packages.ros.org/ros/ubuntu bionic main" > /etc/apt/sources.list.d/ros1-latest.list

RUN apt-get update \
 && apt-get upgrade -y \
 && apt-get install -y -q --no-install-recommends \
  ros-melodic-gazebo11-ros-pkgs

I'm fine with closing this Feature Request if you think it's not a useful image.

ruffsl commented 4 years ago

Yes, again the cleanest way for image size would be to start with new base FROM ubuntu and setup the osrf and ros package repositories and then install ros-${ROS_DISTRO}-gazebo${GAZEBO_VERSION}-pkgs

Wouldn't say you'd have to resort to the ubuntu base image and doing things from scratch, as just editing the gazebo Dockerfiles to build FROM the library/ros:<distro>-ros-base would be enough, keeping the Dockerfile DRY of repo setup and dev tool installion. I was just noting that the desktop tags already have gazebo installed, so going with a parrent image that includes ros but not yet gazebo would help keep Dockerfiles DRY and images small.

# This is an auto generated Dockerfile for gazebo:gzserver11
# generated from docker_images/create_gzserver_image.Dockerfile.em
+FROM ros:foxy-ros-base-focal
-FROM ubuntu:focal
...

I'm fine with closing this Feature Request if you think it's not a useful image.

I think these kind of custom images are best left for downstream user to create and fork instead. But thanks @moriarty for the suggestion, and feel free to open more feature requests and feedback.