search

replicate / cog-flux

Cog inference for flux models
https://replicate.com/black-forest-labs/flux-dev
Apache License 2.0
272 stars 28 forks source link

reduce build size by mounting cache #32

Open technillogue opened 2 weeks ago

technillogue commented 2 weeks ago

when #17 switched to uninstalling torch then reinstalling prerelease torch, it added cached wheels to the final image. by mounting the pip cache, we speed up builds and reduce final image size

yorickvP commented 2 weeks ago

How about just:

  python_packages:
    - "numpy<2"
    - "einops==0.8.0"
    - "fire >= 0.6.0"
    - "huggingface-hub==0.24.3"
    - "safetensors==0.4.3"
    - "sentencepiece==0.2.0"
    - "transformers==4.43.3"
    - "tokenizers==0.19.1"
    - "protobuf==5.27.2"
    - "diffusers==0.29.2"
    - "loguru==0.7.2"
    - "pybase64==1.4.0"
    - "pydash==8.0.3"
    - https://download.pytorch.org/whl/nightly/cu124/torch-2.6.0.dev20240918%2Bcu124-cp311-cp311-linux_x86_64.whl
    - https://download.pytorch.org/whl/nightly/cu124/torchaudio-2.5.0.dev20240918%2Bcu124-cp311-cp311-linux_x86_64.whl
    - https://download.pytorch.org/whl/nightly/cu124/torchvision-0.20.0.dev20240918%2Bcu124-cp311-cp311-linux_x86_64.whl
    - https://download.pytorch.org/whl/nightly/pytorch_triton-3.1.0%2B5fe38ffd73-cp311-cp311-linux_x86_64.whl
daanelson commented 2 weeks ago

good stuff, I was hastily installing nightlies here and made some incorrect assumptions about how we deal with base images. @yorickvP think that's a lot cleaner; let's test that and merge if all looks good.