tensorchord / openmodelz

One-click machine learning deployment (LLM, text-to-image and so on) at scale on any cluster (GCP, AWS, Lambda labs, your home lab, or even a single machine).
https://docs.open.modelz.ai
Apache License 2.0
239 stars 23 forks source link

bug: connection 504 gateway timeout #123

Open Zheaoli opened 1 year ago

Zheaoli commented 1 year ago

I follow the step in the documentation Create your first OpenAI compatible API server

Here's the list info detail

╰─ mdz list
 NAME      ENDPOINT                                                    STATUS   INVOCATIONS  REPLICAS
 jupyter   http://jupyter-2j8g8a2w664nwlsu.192.168.0.239.modelz.live   Scaling            5  0/1
           http://localhost:80/inference/jupyter.default
 jupyter1  http://jupyter1-s37e8r27fc0bt7ap.192.168.0.239.modelz.live  Ready            187  1/1
           http://localhost:80/inference/jupyter1.default

Then when I access the deployment by using endpoint, the error message has been raised

{"message":"no addresses for \"mdz-jupyter.default\"","request":"GET /inference/jupyter.default/","op":"inference-proxy","error":{}}

I guess this is related to the scaling status, but I can't find the root cause by using the command mdz logs jupyter

gaocegege commented 1 year ago

We should add a doc abot the status.

cybergeekgyan commented 1 year ago

Can you assign this issue to me, I would like to work on this issue and resolve with adding documentation and requiured code logic for the same... @gaocegege

gaocegege commented 1 year ago

Sure, thanks! Currently I think we could just add the doc to tell users please use the first endpoint (xxx.modelz.live)