su77ungr / CASALIOY

♾️ toolkit for air-gapped LLMs on consumer-grade hardware
Apache License 2.0
230 stars 31 forks source link

Reduce docker :stable image size #68

Closed su77ungr closed 1 year ago

su77ungr commented 1 year ago

Problem:

hippalectryon-0 commented 1 year ago

I'm not sure I understand exactly what the issue is ?

su77ungr commented 1 year ago

Docker image size is 7GB for some reason https://hub.docker.com/layers/su77ungr/casalioy/stable/images/sha256-4d06875414aa217f3254e51ad589077495e6e14f9ef972044d07422c9b480813?context=explore

Is this caused by cuda tools?

hippalectryon-0 commented 1 year ago

7gb is too much, but a part is probably due to cuda. We can add it as optional in the poetry file.

hippalectryon-0 commented 1 year ago

From https://github.com/hippalectryon-0/CASALIOY/tree/better-docker

user0@DESKTOP-Q3PBFPO:~/CASALIOY$ docker image history casalioy:latest 
IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
fda593445e09   42 seconds ago   /bin/sh -c #(nop) COPY file:648de41f54afe6f9…   613B             
f67cf731fe98   42 seconds ago   /bin/sh -c pip3 uninstall -y streamlit sente…   72.2MB           
579591ecff1c   54 seconds ago   /bin/sh -c pip3 install torch torchvision --…   925MB            
b2b90d314668   2 minutes ago    /bin/sh -c python3 -m poetry install            4.21GB                          
045d81b418d8   4 minutes ago    /bin/sh -c python3 -m poetry config virtuale…   248kB                           
23f73c9e9eaa   4 minutes ago    /bin/sh -c pip3 install poetry                  100MB                           
7c6a0ec932e0   4 minutes ago    /bin/sh -c git checkout better-docker           4.23kB                          
7f40ef97490c   5 minutes ago    /bin/sh -c #(nop) WORKDIR /srv/CASALIOY         0B
hippalectryon-0 commented 1 year ago

REPOSITORY TAG IMAGE ID CREATED SIZE casalioy latest 46a3bc99c3cd 5 minutes ago 2.57GB

We're getting there !

su77ungr commented 1 year ago

sub 3 sounds reasonable. nice