search

eclipse-ankaios / ankaios

Eclipse Ankaios provides workload and container orchestration for automotive High Performance Computing (HPC) software.
https://eclipse-ankaios.github.io/ankaios/
Apache License 2.0
60 stars 18 forks source link

Create smaller devcontainer #89

Closed inf17101 closed 2 weeks ago

inf17101 commented 9 months ago

Description

The devcontainer has been grown a lot: image

This devcontainer is pulled in CI/CD pipeline and also local builds are running very long.

The most dependencies are installed in the Dockerfile.base (the devcontainer base image https://github.com/eclipse-ankaios/ankaios/blob/main/.devcontainer/Dockerfile.base). This is getting to big.

Goals

Having a slim devcontainer image.

Final result

Summary

To be filled when the final solution is sketched.

Tasks

inf17101 commented 9 months ago

I have already started to try to fetch the dependencies to an Alpine image (just to see if it has a big effect when using one of the smallest available images). In general, it works to install the most stuff in alpine (But, I am not 100% finished). It does not mean that the final image must built on top of alpine base image.

The microsoft devcontainer base image just with Rust installed takes 2.97GB (that is too much for a base image).

inf17101 commented 9 months ago

The debian 11-slim is also a very slim base image (and is not alpine). It has only 80MB.

docker pull docker.io/debian:11-slim

image

windsource commented 9 months ago

I think first we need to analyze the disk usage of the dev container to find out the biggest chunks. ncdu is very suited for that.

ncdu -x /

There we can see for ghcr.io/eclipse-ankaios/devcontainer-base:0.6.4:

--- / -------------------------------------------------------------------------------------------------------------------------------------------------------
    4.0 GiB [######################] /usr
  634.0 MiB [###                   ] /home
   44.6 MiB [                      ] /root

and

--- /usr ----------------------------------------------------------------------------------------------------------------------------------------------------
                                     /..
    1.7 GiB [######################] /lib
    1.5 GiB [###################   ] /local
  444.3 MiB [#####                 ] /share
  228.2 MiB [##                    ] /bin

and

--- /usr/lib ------------------------------------------------------------------------------------------------------------------------------------------------
                                     /..
  711.2 MiB [######################] /x86_64-linux-gnu
  360.3 MiB [###########           ] /llvm-14
  183.4 MiB [#####                 ] /jvm
  169.0 MiB [#####                 ] /python3.11
  119.3 MiB [###                   ] /gcc
   67.6 MiB [##                    ] /gcc-cross

and

--- /usr/local ----------------------------------------------------------------------------------------------------------------------------------------------
                                     /..
  954.3 MiB [######################] /rustup
  275.1 MiB [######                ] /cargo
  148.5 MiB [###                   ] /lib
  108.5 MiB [##                    ] /libexec

We can try to install all the things we need based on debian:slim but I fear that we need nearly as much when installing all tools and toolchains we need. But let's try.

krucod3 commented 8 months ago

The base and the dev containers can also be build within the CI/CD pipeline. Take a look here for some inspiration: #116

inf17101 commented 3 months ago

The cargo install overhead can be reduced when using https://crates.io/crates/cargo-binstall/1.4.3. But it searches the crate repositories for release artifacts and downloads it. If they do not exist, then a fallback to the slow cargo install is done. And maintainers must include configuration to support the search of the bininstall tool. I think it is an alternative to avoid the compilation overhead, but the two points I have mentioned are disadvantages...

inf17101 commented 2 months ago

I have tested quickly with a different and smaller base image ubuntu:24.04 (only 80 Mb size) for the Ankaios base devcontainer and with installing the Rust (x86 and aarch64 linux musl) targets directly by adding the targets as arguments when calling the Rust install script (then no extra layer with 'rustup add' is needed afterwards). But it saves only ~900Mb total size. If this ticket is taken, I think we will need a different devcontainer setup to save more size and to be more efficient in the CI/CD pipeline.

windsource commented 2 weeks ago

I addition to the size, our devcontainer also lacks flexibility to install different Rust versions. Currently we use mcr.microsoft.com/devcontainers/rustas base image but it does not support patch version updates e.g. for Rust 18.80.1.

windsource commented 2 weeks ago

Closing as completed.