search

ray-project / ray-llm

RayLLM - LLMs on Ray
https://aviary.anyscale.com
Apache License 2.0
1.22k stars 89 forks source link

Deploy 2 models via `aviary run` - `aviary model list` only displays 1 #30

Closed brent-anyscale closed 1 year ago

brent-anyscale commented 1 year ago

If I deploy 2 models with separate aviary run commands, aviary model list only displays the last one deployed - however in the ray dashboard, it looks like two are running.

Is it possible to deploy two models with separate aviary run commands? If it is, aviary model list should display both models

Yard1 commented 1 year ago

Running aviary run multiple times is currently not supported. Do aviary run --model MODEL --model MODEL.

This will be fixed in the future

AmeerHajAli commented 1 year ago

From Robin: They are not able to query these two models.

robin-anyscale commented 1 year ago

@brent-anyscale can you please confirm, are they able to query the two running models?

avnishn commented 1 year ago

This is now resolved.