search

3x3cut0r / docker

several docker containers
https://github.com/3x3cut0r/docker
23 stars 9 forks source link

9GB download for every `docker compose up` #20

Open bilogic opened 2 months ago

bilogic commented 2 months ago

Hi,

There is this 9GB download each time I run docker compose up which I assume is a LLM?

Why not move this into the docker-compose.yaml examples on https://hub.docker.com/r/3x3cut0r/privategpt#docker-compose

Assuming it is a LLM, this file should persist on the host file system, and will be downloaded only if it does not exists.

Better right?

3x3cut0r commented 1 month ago

If i create this stack i do not have do download the LLM again. Could you provide your docker-compose.yml that i can have a look on it?

bilogic commented 1 month ago

Hi,

I tested it a bit deeper, the redownload only happens if I run docker system prune --volumes -f -a when the container is not running. I think the bigger question is, is this 9GB really all code or should we store it in the data folder?

3x3cut0r commented 1 month ago

its not code ... its the default LLM and its stored at '/home/worker/app/models/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf' if you want to put it into a volume than feel free to do so. no need to be defined by me or do you get some permission problems?

bilogic commented 1 month ago
  1. But Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf is only 4.9GB in my data folder
  2. When I run docker system prune --volumes -f -a, Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf still exists on my data folder, so why up still need to download again?
  3. See below, my up has a 9.063GB download 64142b05a920 
    
[+] Running 9/17
⠙ privategpt [⣿⣿⣿⣿⣿⣿⣿⣦⣿⣿⠀⠀⠀⠀⠀⠀] Pulling                                                                                                                                                15.2s
✔ a2318d6c47ec Already exists                                                                                                                                                         0.0s
✔ 40d734479f14 Pull complete                                                                                                                                                          1.1s
✔ 0b61b7b259eb Pull complete                                                                                                                                                          2.2s
✔ 081a3493c0e7 Pull complete                                                                                                                                                          2.2s
✔ 9ba21e5fdc5c Pull complete                                                                                                                                                          2.5s
✔ dd58e8c66e60 Pull complete                                                                                                                                                          2.5s
✔ b94476f1e614 Pull complete                                                                                                                                                         10.9s
⠼ 9c4fc54ecc6b Downloading [================================>                  ]  144.8MB/222.9MB                                                                                    11.5s
✔ 89e5a37135f1 Download complete                                                                                                                                                      3.1s
✔ 96bb6f2e6429 Download complete                                                                                                                                                      4.3s
⠼ 64142b05a920 Downloading [>                                                  ]  40.77MB/9.063GB                                                                                    11.5s
⠼ f0d70c17e50e Downloading [===>                                               ]  5.387MB/71MB                                                                                       11.5s
⠼ 14f2aa8c4cd9 Waiting                                                                                                                                                               11.5s
⠼ b92d1cd2ce25 Waiting                                                                                                                                                               11.5s
⠼ 991b3da41cb6 Waiting                                                                                                                                                               11.5s
⠼ ef7ee13a3b0b Waiting                                                                                                                                                               11.5s