search

ray-project / ray-llm

RayLLM - LLMs on Ray
https://aviary.anyscale.com
Apache License 2.0
1.22k stars 89 forks source link

How can we use hosted version by making an api call? #22

Closed sanjay-nit closed 1 year ago

sanjay-nit commented 1 year ago

It is possible to make requests to hosted version server?

Yard1 commented 1 year ago

If you mean the version that we are hosting on aviary.anyscale.com, we are not allowing that. You can either deploy aviary yourself, or contact aviary@anyscale.com for more information regarding hosting on Anyscale.

sanjay-nit commented 1 year ago

Thanks for your reply @Yard1 . One more question, Can we deploy aviary LLM models on banana.dev GPUs. TIA.

Yard1 commented 1 year ago

I haven't used banana.dev myself but as long as you can run Ray on those (and you should be able to), I don't see why not