Prepare containers for Docker Hub release

hidmic commented 3 months ago

Proposed changes

This patch:

fixes builds from Focal thru Noble
forwards args across Earthfiles
allows tag renaming

Type of change

[ ] 🐛 Bugfix (change which fixes an issue)
[x] 🚀 Feature (change which adds functionality)
[ ] 📚 Documentation (change which fixes or extends documentation)

Checklist

[ ] Lint and unit tests (if any) pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

hidmic commented 3 months ago

With this patch I managed to build images for Noetic, Humble, Iron, Jazzy, and Rolling locally.

glpuga commented 3 months ago

I have no access to the private dockerhub images.

Screenshot from 2024-07-11 22-44-46

hidmic commented 3 months ago

@glpuga those images are not in any registry, they have to be built locally, and for that you need to use earthly. docker-compose cannot build them automatically, so it's necessarily a two step process.

In this case,

cd lambkin
./tools/setup.sh  # to bootstrap tooling
./tools/earthly ./src/benchmarks/beluga_vs_nav2+ubuntu-local-devel

The tool itself is nicely documented (https://docs.earthly.dev/basics). Eventually I want to document targets so that ./tools/earthly doc yields useful output, but I haven't dug into that yet.

hidmic commented 3 months ago

Also, note that beluga_vs_nav2 is getting relocated soon.

hidmic commented 3 months ago

Still spotted an issue in devel targets. Fixed it in https://github.com/ekumenlabs/lambkin/pull/86/commits/0db5a1e158cf4db5b084a32d4a731dafb2b18450.

hidmic commented 3 months ago

It was more than one issue. Using --pass-args was negating the caching benefits of all this (see https://github.com/earthly/earthly/issues/4162) and poetry caught some dependency issues in Noble that plain pip had not.

@glpuga PTAL!

marcoshuck commented 3 months ago

If we're publishing images to Dockerhub, I'd suggest we add LABELS to the resulting image. The OpenContainers organization suggests a few labels.

~~We can add our own labels being something like: com.ekumenlabs.<label>.~~

We should use the org.opencontainers.image prefix for the pre-defined annotation keys, and com.ekumenlabs. if we want to add any custom label.

glpuga commented 3 months ago

./tools/setup.sh  # to bootstrap tooling

Now I can build and run the image using the fixed instructions.

I added missing steps to the benchmark readme and fixed others, https://github.com/ekumenlabs/lambkin/blob/f29595bdad72dffed1764b5ae54c8dbb391fdd57/src/benchmarks/beluga_vs_nav2/README.md.

However, once inside the workspace is only writable by root.

Fixing this permission problem live in the container and following the updated instructions in the README step by step I get a "/usr/bin/env: ‘shepherd’: No such file or directory" error when running the benchmark.

glpuga@826862c4fa89:/workspace/src/lambkin/benchmarks/beluga_vs_nav2/playground$ ros2 run beluga_vs_nav2 nominal.robot
/usr/bin/env: ‘shepherd’: No such file or directory
[ros2run]: Process exited with failure 127
glpuga@826862c4fa89:/workspace/src/lambkin/benchmarks/beluga_vs_nav2/playground$

hidmic commented 3 months ago

@marcoshuck labels adjusted for the images that will be pushed.

hidmic commented 3 months ago

However, once inside the workspace is only writable by root.

This is fixed now.

I get a "/usr/bin/env: ‘shepherd’: No such file or directory" error when running the benchmark

This was beluga_vs_nav2 not declaring lambkin-shepherd as a dependency, and thus colcon not building it. Fixing that sent me down a rabbit hole (see https://github.com/ekumenlabs/lambkin/pull/86/files#diff-9b2f68e65b6a2d12c1828a6bc1db8ec65a115e246223867d3b40d50b4896b980R27-R33). I really wonder how ROS folks are going to deal with Python stuff moving forward. colcon support for Python packages and ament_cmake_python both need a major overhaul.

FYI @glpuga

glpuga commented 3 months ago

I'm getting issues updating the system now in both the setup step (in a new machine), and the build step (in the same machine as before, in a diferent network and location). Both look temporary upstream server disruptions. I'll try again later.

glpuga commented 3 months ago

I pushed some more fixes to the README. With this configuration I can get it to build, run end-to-end and generate a report, so it's good to merge.

hidmic commented 3 months ago

Merging in the interest of time. Thanks for all the reviews and testing :heart:

Ekumen-OS / lambkin