search

skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.8k stars 509 forks source link

[SkyServe] Cache output of `sky serve status --endpoint` #2915

Open cblmemo opened 10 months ago

cblmemo commented 10 months ago

Currently, every time we run sky serve status --endpoint, it will contact the sky serve controller to get the endpoint. It is not friendly for a bash-based script that calls this CLI multiple times, say curl $(sky serve status --endpoint svc1)/v1/models. We should have a local database serve as a cache for the service endpoint since it will not change while the service is running.

Notice that we still use the remote database on the SkyServe controller as the only source of truth. If we don't find the service endpoint in the local cache, then we contact the SkyServe controller for the endpoint.

github-actions[bot] commented 6 months ago

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.