Download the model at Docker image build time

[iosifnicolae2]

What happened?

We would like to be able to download the model at build time and then reference the model from the filesystem.

What Python version are you on? e.g. python --version

Python 3.11

Version

0.2.7 (Latest)

What os are you seeing the problem on?

Linux

Relevant stack traces and/or logs

No response

[Anush008]

You can try

RUN pip install fastembed

RUN python3 -c "from fastembed import TextEmbedding;TextEmbedding(model_name="BAAI/bge-small-en-v1.5")"

The model will be cached in the default location. You won't need to do any additional config in the code.

[iosifnicolae2]

yes, but we would like to make the filesystem read-only when deploying the container in kubernetes, and when the library is adding a lock it will fail

[I8dNLo]

Hi! You actually have a way to do it without any changes in the lib. Just use cache_dir in TextEmbedding or ImageEmbedding. Here is an example of doing it.

offline_docker_run.py

from fastembed import TextEmbedding
from typing import List
import os

FASTEMBED_CACHE_DIR = os.environ.get('FASTEMBED_CACHE_DIR')
documents: List[str] = [
    "This is built to be faster and lighter than other embedding libraries e.g. Transformers, Sentence-Transformers, etc.",
    "fastembed is supported by and maintained by Qdrant.",
]

embedding_model = TextEmbedding(cache_dir=FASTEMBED_CACHE_DIR)
print("The model BAAI/bge-small-en-v1.5 is ready to use.")

embeddings_generator = embedding_model.embed(documents)  # reminder this is a generator
embeddings_list = list(embedding_model.embed(documents))
print(len(embeddings_list[0])) # Vector of 384 dimensions

Dockerfile

FROM python:3.11-slim

ARG FASTEMBED_CACHE_DIR
ENV FASTEMBED_CACHE_DIR=${FASTEMBED_CACHE_DIR}

RUN apt-get update && apt-get install -y \
    gcc \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN mkdir -p ${FASTEMBED_CACHE_DIR} && ls -la ${FASTEMBED_CACHE_DIR}

COPY ${FASTEMBED_CACHE_DIR} /app/${FASTEMBED_CACHE_DIR}

WORKDIR /app
COPY experiments/offline_docker_run.py .

RUN pip install fastembed

CMD ["python", "offline_docker_run.py"]

Build command: docker build --build-arg FASTEMBED_CACHE_DIR=my_cache_dir -t fastembed_offline . Run command: docker run --network none -it fastembed_offline