search

nix-community / poetry2nix

Convert poetry projects to nix automagically [maintainer=@adisbladis,@cpcloud]
MIT License
835 stars 435 forks source link

How to create a "minimal" application/container? #1790

Open naddeoa opened 1 week ago

naddeoa commented 1 week ago

Describe the issue

I have a repo up with my work for reference: https://github.com/naddeoa/nix-python-minimal-container-issue

This repo has a small python application with a few heavy dependencies (things that have native code/ require gcc, etc). When I create a container using buildLayeredImage the resulting image comes out to 415mb. When I build the same stuff using a naive Docckerfile then I get an image of size 254mb.

I know there's another issue about extra pyc files in the derivations, and that is definitely an issue, but it doesn't explain the entire difference. I can delete those pyc files and get the image that I generate with buildLayeredImage down to 281mb, and it's still a fair bit bigger than the Dockerfile version while the Dockerfile version has a bunch of junk in it like perl. Looking at the deployed layers, I can see things like and gcc sticking around even though they're not runtime dependencies. It was probably originally pulled in by my poetry dependency on faster_fifo. It seems like every dependency in the site-packages folder of the buildLayeredImage image is slightly bigger even without the pyc files. For example, pandas is 46mb instead of 43mb in the Dockerfile image, numpy is 30mb instead of 29mb, and so on.

Am I doing something wrong on the container construction side or is this not something that poetry2nix can actually solve generically?

Additional context

All files are here: https://github.com/naddeoa/nix-python-minimal-container-issue/tree/master

declension commented 1 week ago

For reference the other issue: #1580

naddeoa commented 1 week ago

This one might also be related: https://github.com/nix-community/poetry2nix/issues/1616

naddeoa commented 1 week ago

As a side note, maybe we just don't care about making the image minimal because nix really can't do a perfect job there anyway? I've been playing around with Slim and for this little test I can get the image size down to a couple dozen megabytes. Even if I could make a more minimal image with poetry2nix, it would never be able to compensate for people being bad at specifying their requirements in the python ecosystem (very common) and I would still use a tool like Slim for minification when it really mattered.

Funnily enough, Slim's output on the poetry2nix container is still a few mb larger than Slim's output on the Dockerfile generated container.