datamade / how-to

📚 Doing all sorts of things, the DataMade way
MIT License
86 stars 12 forks source link

Optimizing Docker images #104

Open hancush opened 4 years ago

hancush commented 4 years ago

Background

We have great (and conventional, AFAICT) patterns for creating Docker images from our Django applications. The downside is that these images usually clock in at 1 GB in size or more. This can really start to add up, especially when we're working with constrained server space (e.g., https://github.com/datamade/la-metro-dashboard/issues/36).

Proposal

As I mentioned, our containerized Django apps seem to follow convention, but I'm curious if there are things we can do to reduce the size. Some ideas include using a smaller base image, omitting collected static files (by collecting static in the entry point, instead of in the build), and/or leveraging remote static files storage so large static files aren't included in the application directory.

I'd like to spend a day researching and trialing strategies for optimizing Docker image size, specifically for our Django applications.

Deliverables

The deliverable for this will either be "Looks like our current strategy is the best", or updates to our Docker templates. I'll also keep a mission log of the strategies I try, for future reference.

Timeline

I expect this can be done in about a day.

smcalilly commented 7 months ago

@antidipyramid has done some multi-stage steps on the workplace project, he'll replicate this work for the new Block Builder map.

We'll need to add this to the cookiecutter once we get it figured out (maybe? projects have different image requirements).

Is there a django based image? Yes, but it's deprecated.

smcalilly commented 4 months ago

Related blog post: https://blacksmith.sh/blog/cache-is-king-a-guide-for-docker-layer-caching-in-github-actions