Closed chaudhariatul closed 2 months ago
Hello!
Indeed, Sentence Transformers has a few direct dependencies and a good amount of indirect dependencies. For reference, these are the direct ones at this time: https://github.com/UKPLab/sentence-transformers/blob/add421f21508cd2baf4cd32af31624c63b355a1d/pyproject.toml#L32-L41
Out of all of the direct and indirect dependencies, torch
and in particular its (optional dependency) CUDA use the most disk space. On Unix devices, the default torch
behaviour is to install with CUDA, whereas on Windows, the default behaviour is to install without CUDA. So, if you're using a Docker image, you're likely installing torch
with CUDA by default. If you're indeed using a GPU, then there's no way around this - you can't really shrink torch
& CUDA. But if you're not using a GPU, then my recommendation is to first install torch
without CUDA (pip install torch --index-url https://download.pytorch.org/whl/cpu
), and then install sentence-transformers. During the ST installation, it will see that torch
is installed, and it'll not install it with CUDA.
Additionally, usually when you install something via pip
, pip
will cache the whl
file as well as installing the Python files for that module. On your PC/Laptop this is usually fine, but on a Docker image you might want to avoid this (as you're usually only installing with pip
once and you don't need that cache). So, you can use the --no-cache-dir
with the pip install
to get a smaller image: https://pip.pypa.io/en/stable/cli/pip/#cmdoption-no-cache-dir.
Here's an indication of the difference in filesizes excluding the unnecessary cache (note, this is on Windows, so there's no Triton):
With CUDA: 5.3GB
Without CUDA: 1.6GB
So, if you're not using a GPU, then my recommendation is:
pip install --no-cache-dir -U torch --index-url https://download.pytorch.org/whl/cpu
pip install --no-cache-dir sentence-transformers
And your Docker image should shrink by a good amount. If you are using a GPU, then there's not that much that can be done other than not using a cache:
pip install --no-cache-dir sentence-transformers
And to answer your proposal more concretely: I can't make any of the direct dependencies optional (except maybe Pillow/PIL which is ~7MB) as they are all crucial to the core functionality of Sentence Transformers.
This worked! Thank you Tom!
Feature request
Request to remove hard dependencies that require CUDA and OpenAI python packages with
sentence-transformers
python package.Motivation
Many python packages default to use CUDA and OpenAI and so when developing with non CUDA devices and using different embedding models there are large number of dependencies downloaded.
A docker image created with these requirements often uses upto 9GB and a local python environment uses 6GB or more.
$ pip install sentence-transformers
A reduce storage footprint will allow the following:
Your contribution
I'll test a combination of packages that can help reduce storage footprint.
Link: https://github.com/huggingface/transformers/issues/32904