make scheduler metrics visible

sanposhiho commented 2 years ago

It would be nice if we could see the time used in scheduling for each plugin.

[ ] ~~Aggregate the time used for each plugin.~~
- ~~It can be achieved in the same way as scheduling results. We can aggregate it in simulatorPlugin and send used time to resultStore.~~
- EDIT: Not need to aggregate the time on simulator. We can use plugin_execution_duration_seconds to see each plugins time.
[ ] show them on WebUI in table or graph..? should be something nice.
[ ] update doc. (include https://github.com/kubernetes-sigs/kube-scheduler-simulator/blob/master/docs/how-it-works.md)

/assign /kind feature

196Ikuchil commented 2 years ago

Hi @sanposhiho Is there any progress on this? Are you looking for a worker?

sanposhiho commented 2 years ago

I haven't worked on this. So, feel free to pick this issue. But, I think we should work on the following issue before this feature. https://github.com/kubernetes-sigs/kube-scheduler-simulator/issues/125

matthewygf commented 2 years ago

Hi @196Ikuchil @sanposhiho , I am planning to work on this feature, additionally, my thoughts are to do something similar to Kube-Scheduler, where these performance metrics are emitted in prometheus metrics format. Something like this ? WDYT ?

EDIT: turns out it is already implemented here. Maybe we could implement a metrics endpoint for the simulator server ?

sanposhiho commented 2 years ago

EDIT: turns out it is already implemented here. Maybe we could implement a metrics endpoint for the simulator server ?

Oh, yes. You are right. I misunderstood about the metrics plugin_execution_duration_seconds 😓 Change this issue to make scheduler metrics visible.

/retitle make scheduler metrics visible

sanposhiho commented 2 years ago

One problem is pluginMetricsSamplePercent is const and we cannot change the value. Ideally, I would like to see metrics recorded for all pod schedules. (Because unlike real clusters, users do not always create many pods on the simulator)

I will make a feature request issue to k/k later. We will not know if the request will be approved until discussing it, but consider a work-around or another way if it is not approved then.

sanposhiho commented 2 years ago

https://github.com/kubernetes/kubernetes/issues/108903

matthewygf commented 2 years ago

Regarding the metrics available, should we add a metric handler to the simulator server ? WDYT about having our own frontend charts to show these metrics like dashboard (probably need to investigate some of the frontend charts) or using prometheus to show them ?

sanposhiho commented 2 years ago

@matthewygf

Sorry, I missed your comment.

having our own frontend charts to show these metrics like dashboard (probably need to investigate some of the frontend charts) or using prometheus to show them ?

Yeah, hmm, I prefer the former. But, charts in frontend.. It could be a very tough road because at least I'm not very familiar with frontend... 😓 It would be so much easier if there were a public component or a public library to show charts from data in the form of grafana.

matthewygf commented 2 years ago

@matthewygf

Sorry, I missed your comment.

having our own frontend charts to show these metrics like dashboard (probably need to investigate some of the frontend charts) or using prometheus to show them ?

Yeah, hmm, I prefer the former. But, charts in frontend.. It could be a very tough road because at least I'm not very familiar with frontend... 😓 It would be so much easier if there were a public component or a public library to show charts from data in the form of grafana.

No problem @sanposhiho , I will have a look at it ~

sanposhiho commented 2 years ago

One approach is to process the grafana data on the simulator api into a form that is easy to display on the front end, and then display it on the front end using some library or component.

It seems vuetify has the component to write a simple graph. (But, not sure whether the component is sufficiently expressive) https://vuetifyjs.com/en/components/sparklines/

sanposhiho commented 2 years ago

/triage accepted Not to mark as stale.

sanposhiho commented 2 years ago

/area simulator

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle stale
Mark this issue or PR as rotten with /lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue or PR as fresh with /remove-lifecycle rotten
Close this issue or PR with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

sanposhiho commented 1 year ago

/remove-lifecycle rotten /lifecycle frozen

kubernetes-sigs / kube-scheduler-simulator

make scheduler metrics visible #60