michaelfeil / infinity

Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting a wide range of text-embedding models and frameworks.
https://michaelfeil.eu/infinity/
MIT License
959 stars 71 forks source link

test #264

Closed michaelfeil closed 2 weeks ago

michaelfeil commented 2 weeks ago

System Info

test

Information

Tasks

Reproduction

How do i install infinity?

Expected behavior

-

greptile-apps[bot] commented 2 weeks ago

To install Infinity, follow these steps:

  1. Using Docker:
    
    port=7997
    model1=michaelfeil/bge-small-en-v1.5
    model2=mixedbread-ai/mxbai-rerank-xsmall-v1
    volume=$PWD/data

docker run -it --gpus all \ -v $volume:/app/.cache \ -p $port:$port \ michaelf34/infinity:latest \ v2 \ --model-id $model1 \ --model-id $model2 \ --port $port


2. **Using CLI via pip**:
```bash
pip install infinity-emb[all]
infinity_emb v2 --model-id BAAI/bge-small-en-v1.5

Refer to the README.md and deploy.md for more details.

References

/README.md /docs/docs/deploy.md /docs/docs/index.md

Ask Greptile