search

deepinfra / deepctl

Command line tool for Deep Infra cloud ML inference service
Apache License 2.0
23 stars 0 forks source link

Is it possible to run Sentence transformes? #4

Closed fugaprascolinas closed 11 months ago

fugaprascolinas commented 1 year ago

This library runs the basis for different vector databases and depends in gpu for speed and stability. I am running it in Ec2 using only cpu, but the results are bad.

NikolaBorisov commented 1 year ago

@fugaprascolinas We have support for number of sentance transformers models. You can quickly try them like this:

deepctl infer \
    -m 'sentence-transformers/all-MiniLM-L12-v2'  \
    -i 'inputs=["I like chocolate"]

Here is a list of the models we have right now: https://stage.deepinfra.com/models?type=embeddings If you want any of the models on hugging face we can easily add them https://huggingface.co/models?pipeline_tag=sentence-similarity&sort=downloads

If you want to deploy your own custom model let me know, we are working on support for this, should be out soon.

NikolaBorisov commented 1 year ago

@fugaprascolinas let me know if this helper. I would like to close this issue.