search

kubernetes-sigs / wg-serving

WG Serving
https://github.com/kubernetes/community/tree/master/wg-serving
Apache License 2.0
13 stars 12 forks source link

Configure prometheus_port for both jetstream jax and pytorch servers #24

Closed JeffLuoo closed 4 weeks ago

JeffLuoo commented 4 weeks ago

The prometheus_port for pytorch server in jetstream is not documented but it's available starting v0.2.2: https://github.com/AI-Hypercomputer/JetStream/releases/tag/v0.2.2

The port number 9100 is referenced from the benchmark tool: https://github.com/GoogleCloudPlatform/ai-on-gke/blob/main/benchmarks/inference-server/jetstream/jetstream.yaml

Please let me know if another port number is preferred.

JeffLuoo commented 4 weeks ago

cc: @jjk-g for review.

jjk-g commented 4 weeks ago

Thanks for the addition and the references!

/approve /lgtm

k8s-ci-robot commented 4 weeks ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JeffLuoo, jjk-g

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[serving-catalog/OWNERS](https://github.com/kubernetes-sigs/wg-serving/blob/main/serving-catalog/OWNERS)~~ [jjk-g] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment