search

bitcraze / docker-builder

A docker images used for building modules
MIT License
4 stars 2 forks source link

Reduce size of Docker image #20

Open gemenerik opened 1 month ago

gemenerik commented 1 month ago

The built Docker image is quite large, leading to slower CI processes due to frequent pulls. In fact, it is so big that GitHub's caching features are largely unsupported. Some features of the image are rarely used (esp-idf). To improve efficiency and sustainability, consider implementing multi-stage builds, separate builder images for different projects, using a smaller base image, or cleaning up unnecessary dependencies. This would enhance build times and reduce resource usage.

gemenerik commented 1 month ago

An alternative option is to run more of the CI locally

gemenerik commented 1 month ago

One approach is to stop using the Docker builder images for our GitHub Actions.

One example of what this could look like is https://github.com/bitcraze/crazyflie-lib-python/pull/494.

One significant advantage of this transition is the elimination of the need to pull the large (currently 5.41 GB for bitcraze/builder, 977.51 MB for bitcraze/web-builder) Docker image for every CI step. In cflib, we download 6.39 GB of Docker image per CI run. For crazyflie-firmware ( with 24 pulls of the builder) this amounts to 129.84 GB per CI run. Let's hope GitHub does some automatic caching for us. This change should lead to faster and more sustainable CI runs. By leveraging GitHub's default environments, we gain flexibility in dependency management, which may help us adapt more easily to changing requirements.

However, this shift raises important considerations. If we choose to continue using monolithic Docker images for local development, we will need to ensure that these images are tested (e.g. in our CI pipeline) as well. While these Docker images provide a consistent environment for users to build all projects, maintaining them has become challenging. Compatibility issues can arise, especially when certain components or compilers require different dependencies, complicating the build process.