search

skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.43k stars 456 forks source link

[Serve] Expose multiple ports while using sky serve #3727

Open aswanthkrishna opened 1 month ago

aswanthkrishna commented 1 month ago

expose multiple ports while using sky serve

I would like the ability to add multiple ports in the resource definition in sky serve. This might be especially useful when i want to deploy applications with a separate frontend and backed server.

Jayasankar-kk commented 1 month ago

Serving without mentioning ports, We are unable to serve without mentioning ports under resources. I think this is also a similar issue

When I tried sky serve with lambda cloud I got the following error "ValueError: No enabled clouds support opening ports. To fix: do not specify resources.ports, or enable a cloud that does support this feature." When I tried serving without mentioning the ports in the yaml file, I got the following error "ValueError: Must only specify one port in resources. Each replica will use the port specified as application ingress port."

info-wordcab commented 1 month ago

@cblmemo Following up to what Adrian mentioned in this issue, this works if you comment out this error in cli.py, and this error in serve/core.py, with the caveat that the lowest port is selected. So if your FastAPI server is on 5001, and Grafana is on 3000, you'd switch Grafana to 5005 or something higher.