Remote address refuse queries

ray-project / ray-llm

RayLLM - LLMs on Ray

https://aviary.anyscale.com

Apache License 2.0

1.22k stars 87 forks source link

Remote address refuse queries #100

Open rifkybujana opened 9 months ago

rifkybujana commented 9 months ago

Hi, I've managed to deploy my models using AWS as instructed in the README.md. It works well by using the http://localhost:8000/v1 endpoint inside the cluster. However, when I use http://<remote head server public ip>:8000/v1, it always refuses connection to any query. There are no problems accessing the dashboard or Prometheus using the same public IP. Do you know what I am doing wrong?

akshay-anyscale commented 8 months ago

Can you provide the code you are using for querying?