search

skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.62k stars 483 forks source link

[SkyServe] controller resources in service YAML #2916

Open cblmemo opened 9 months ago

cblmemo commented 9 months ago

According to a user's request, we should have a controller_resources section under service to override the specification in ~/.sky/config.yaml. If both specified controller resources, what is specified in the service yaml takes precedence.

landscapepainter commented 8 months ago

+1 This would be really useful especially for the new HeteroGPU serving policy, #2927, which may run launch processes more than our default controller spec can handle.

dtran24 commented 8 months ago

If both controller resources are specified, would the controller sources under service be a complete or partial override of the specification in ~/.sky/config.yaml?

cblmemo commented 8 months ago

If both controller resources are specified, would the controller sources under service be a complete or partial override of the specification in ~/.sky/config.yaml?

IMO it should be a partial override; though we could discuss it in our coming meeting :)

cblmemo commented 8 months ago

After an offline discussion, we decided to defer that since this will complicate our service yaml.

github-actions[bot] commented 4 months ago

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.