tomaarsen
commented
2 months ago Hello!
Indeed, Sentence Transformers has a few direct dependencies and a good amount of indirect dependencies. For reference, these are the direct ones at this time: https://github.com/UKPLab/sentence-transformers/blob/add421f21508cd2baf4cd32af31624c63b355a1d/pyproject.toml#L32-L41
Out of all of the direct and indirect dependencies, torch and in particular its (optional dependency) CUDA use the most disk space. On Unix devices, the default torch behaviour is to install with CUDA, whereas on Windows, the default behaviour is to install without CUDA. So, if you're using a Docker image, you're likely installing torch with CUDA by default. If you're indeed using a GPU, then there's no way around this - you can't really shrink torch & CUDA. But if you're not using a GPU, then my recommendation is to first install torch without CUDA (pip install torch --index-url https://download.pytorch.org/whl/cpu), and then install sentence-transformers. During the ST installation, it will see that torch is installed, and it'll not install it with CUDA.
Additionally, usually when you install something via pip, pip will cache the whl file as well as installing the Python files for that module. On your PC/Laptop this is usually fine, but on a Docker image you might want to avoid this (as you're usually only installing with pip once and you don't need that cache). So, you can use the --no-cache-dir with the pip install to get a smaller image: https://pip.pypa.io/en/stable/cli/pip/#cmdoption-no-cache-dir.
Here's an indication of the difference in filesizes excluding the unnecessary cache (note, this is on Windows, so there's no Triton):
With CUDA: 5.3GB
Without CUDA: 1.6GB
So, if you're not using a GPU, then my recommendation is:
pip install --no-cache-dir -U torch --index-url https://download.pytorch.org/whl/cpu
pip install --no-cache-dir sentence-transformersAnd your Docker image should shrink by a good amount. If you are using a GPU, then there's not that much that can be done other than not using a cache:
pip install --no-cache-dir sentence-transformersAnd to answer your proposal more concretely: I can't make any of the direct dependencies optional (except maybe Pillow/PIL which is ~7MB) as they are all crucial to the core functionality of Sentence Transformers.
- Tom Aarsen
chaudhariatul
Feature request
Request to remove hard dependencies that require CUDA and OpenAI python packages with
sentence-transformerspython package.Motivation
Many python packages default to use CUDA and OpenAI and so when developing with non CUDA devices and using different embedding models there are large number of dependencies downloaded.
A docker image created with these requirements often uses upto 9GB and a local python environment uses 6GB or more.
$ pip install sentence-transformersA reduce storage footprint will allow the following:
Your contribution
I'll test a combination of packages that can help reduce storage footprint.
Link: https://github.com/huggingface/transformers/issues/32904