Add `containers/tei/{cpu,gpu}/1.5.0` - Githubissues

huggingface / Google-Cloud-Containers

Including Hugging Face Deep learning Containers for Google Cloud

Apache License 2.0

112 stars 10 forks source link

Add `containers/tei/{cpu,gpu}/1.5.0` #61

Open alvarobartt opened 1 month ago

alvarobartt commented 1 month ago

Description

This PR adds a new container for TEI v1.5.0 recently released (see https://github.com/huggingface/text-embeddings-inference/releases/tag/v1.5.0).

The main features within TEI v1.5.0 are the following:

Added /similarity endpoint for sentence-similarity tagged models in the Hugging Face Hub at https://huggingface.co/models?pipeline_tag=sentence-similarity
Added ONNX runtime for CPU deployments, improving the throughput

To inspect the changes required to make the TEI container work in GCP, see the diff at:

alvarobartt commented 2 days ago

[!NOTE] This PR is on hold, since the CPU version requires the model to have ONNX compatible weights, and there are a bunch of models that only contain the safetensors weights.