Sygil-Dev / stable-diffusion

GNU Affero General Public License v3.0
1.72k stars 149 forks source link

Avoid copying the models, outputs and src when building the image. #285

Closed swfsql closed 2 years ago

swfsql commented 2 years ago

The models, src and outputs are mapped as volumes for docker.

oc013 commented 2 years ago

I think you're mistaken that it gets copied.

swfsql commented 2 years ago

@oc013 After the models have been downloaded, with this change when I run docker-compose up --build only 96MB gets copied as context into the image:

$ docker-compose up --build
Building stable-diffusion
Sending build context to Docker daemon  96.06MB
Step 1/12 ...

Without the volume mapping and having them in .dockerignore, everything gets copied as context:

$ docker-compose up --build      
Removing sd
Building stable-diffusion
Sending build context to Docker daemon  5.504GB
Step 1/12 ...

That initial copying also takes much longer on the later.

oc013 commented 2 years ago

Can you try to update to latest docker and use docker compose and see if you have the same issue? I'm not seeing this. Also if that's an issue it should only require the .dockerignore not the other changes?

=> [internal] load build context                                                                                                                                    0.0s
 => => transferring context: 761.20kB
swfsql commented 2 years ago

@oc013 For docker compose I got the same behavior as I previously had.

$ docker compose version
Docker Compose version v2.3.3

Please note that when testing this, the models must be already downloaded. It makes a difference for when "rebuilding" the images.

Having empty directories committed are indeed unnecessary as docker will create empty directories if they are mounted as volumes, although I'm not sure what permissions those directories will have. I'm using docker in rootless mode so they get created under my normal user, but it's possible that docker running under root will have them owned by root as well.

swfsql commented 2 years ago

I squashed and force-pushed, which should be the same except that no empty directories were force-added. But I see what you meant, as just the .dockerignore should suffice, as the current directory as a whole is being added as a volume on /sd. Will update it shortly.

oc013 commented 2 years ago

I'll just chalk it up to you having a different OS. I don't delete the models or move my output before a rebuild and do not see this. I think only change required to alleviate what you're experiencing should be adding the .dockerignore only, don't need to map extra volumes or change anything else.

swfsql commented 2 years ago

That might be the case. My pc is as follows:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.1 LTS
Release:    22.04
Codename:   jammy

docker --version
Docker version 20.10.14, build a224086

I've force-pushed a squashed commit again, in which basically just the .dockerignore is added. I also removed the outputs mapping as it's already mapped by it's parent (the ./ directory), but as you mentioned, you could prefer to avoid extra changes.

I'm using it personally and thought could be useful for others using docker, if they decide to rebuild. But I'm not sure why you would get a different docker copy behavior for this case.

oc013 commented 2 years ago

I just tested without that mapping, if the outputs volume isn't mapped it won't function as a symlink on the host.