feat: implement scheduler_perf

sanposhiho commented 9 months ago

What type of PR is this?

/kind feature

What this PR does / why we need it:

Implement scheduler_perf for the wasm extension
- The impl is mostly imported from
- Those imported logic should be eliminated after https://github.com/kubernetes/kubernetes/pull/119079
Implement nodenumber extender so that we can have a comparison among wasm, extender, and plugin.

Here is the benchmark result in my environment. The plugin is (of course) not much slower than the default scheduler, the wasm is slower than the plugin, and the extender is the slowest very much.

benchmark result

```json { "version": "v1", "dataItems": [ { "data": { "Average": 991, "Perc50": 991, "Perc90": 991, "Perc95": 991, "Perc99": 991 }, "unit": "pods/s", "labels": { "Metric": "SchedulingThroughput", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2" } }, { "data": { "Average": 4.689029146000004, "Perc50": 2.4709141274238227, "Perc90": 7.844961240310078, "Perc95": 16, "Perc99": 50.666666666666664 }, "unit": "ms", "labels": { "Metric": "scheduler_scheduling_attempt_duration_seconds", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2" } }, { "data": { "Average": 383.4590710599999, "Perc50": 419.6143250688705, "Perc90": 595.9228650137741, "Perc95": 617.961432506887, "Perc99": 635.5922865013774 }, "unit": "ms", "labels": { "Metric": "scheduler_pod_scheduling_duration_seconds", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2" } }, { "data": { "Average": 0.017050759000000013, "Perc50": 0.050200803212851405, "Perc90": 0.09036144578313253, "Perc95": 0.09538152610441766, "Perc99": 0.09939759036144578 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2", "extension_point": "PreFilter" } }, { "data": { "Average": 0.24587377199999982, "Perc50": 0.17065868263473055, "Perc90": 0.38133333333333336, "Perc95": 0.5966101694915255, "Perc99": 1.3 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2", "extension_point": "Filter" } }, { "data": { "Average": 0.033266842999999977, "Perc50": 0.05081300813008131, "Perc90": 0.09146341463414635, "Perc95": 0.09654471544715447, "Perc99": 0.16666666666666669 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2", "extension_point": "PreScore" } }, { "data": { "Average": 0.5242404499999999, "Perc50": 0.31766169154228857, "Perc90": 0.6173228346456693, "Perc95": 0.7748031496062993, "Perc99": 1.5529411764705883 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/Default/500Nodes/namespace-2", "extension_point": "Score" } }, { "data": { "Average": 500, "Perc50": 362, "Perc90": 638, "Perc95": 638, "Perc99": 638 }, "unit": "pods/s", "labels": { "Metric": "SchedulingThroughput", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2" } }, { "data": { "Average": 3.481629186000002, "Perc50": 3.1314285714285717, "Perc90": 4.824742268041237, "Perc95": 6.88659793814433, "Perc99": 32 }, "unit": "ms", "labels": { "Metric": "scheduler_scheduling_attempt_duration_seconds", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2" } }, { "data": { "Average": 655.518858365, "Perc50": 672.7893738140418, "Perc90": 1158.5578747628085, "Perc95": 1219.2789373814041, "Perc99": 1267.855787476281 }, "unit": "ms", "labels": { "Metric": "scheduler_pod_scheduling_duration_seconds", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2" } }, { "data": { "Average": 0.020502628999999998, "Perc50": 0.05025125628140704, "Perc90": 0.09045226130653267, "Perc95": 0.09547738693467336, "Perc99": 0.09949748743718594 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2", "extension_point": "PreFilter" } }, { "data": { "Average": 0.18554582299999992, "Perc50": 0.16421636615811375, "Perc90": 0.3434343434343435, "Perc95": 0.393939393939394, "Perc99": 0.88 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2", "extension_point": "Filter" } }, { "data": { "Average": 0.115504651, "Perc50": 0.055928411633109625, "Perc90": 0.10857142857142858, "Perc95": 0.18000000000000002, "Perc99": 1.375 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2", "extension_point": "PreScore" } }, { "data": { "Average": 0.9955440430000002, "Perc50": 0.9922580645161292, "Perc90": 1.5083870967741937, "Perc95": 1.5729032258064517, "Perc99": 2.9217391304347826 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/WasmOnly/500Nodes/namespace-2", "extension_point": "Score" } }, { "data": { "Average": 111.11111111111111, "Perc50": 121, "Perc90": 124, "Perc95": 124, "Perc99": 124 }, "unit": "pods/s", "labels": { "Metric": "SchedulingThroughput", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2" } }, { "data": { "Average": 10.0189723, "Perc50": 12.214963119072708, "Perc90": 15.586933614330874, "Perc95": 16.340425531914892, "Perc99": 29.95744680851064 }, "unit": "ms", "labels": { "Metric": "scheduler_scheduling_attempt_duration_seconds", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2" } }, { "data": { "Average": 4185.379899980998, "Perc50": 4193.374233128834, "Perc90": 8899.685863874347, "Perc95": 9569.842931937173, "Perc99": 10105.968586387435 }, "unit": "ms", "labels": { "Metric": "scheduler_pod_scheduling_duration_seconds", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2" } }, { "data": { "Average": 0.014909018000000008, "Perc50": 0.05015045135406219, "Perc90": 0.09027081243731194, "Perc95": 0.09528585757271817, "Perc99": 0.09929789368104314 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2", "extension_point": "PreFilter" } }, { "data": { "Average": 0.17747714799999997, "Perc50": 0.16090133982947627, "Perc90": 0.2969325153374233, "Perc95": 0.35828220858895704, "Perc99": 0.5846153846153846 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2", "extension_point": "Filter" } }, { "data": { "Average": 0.027234693000000004, "Perc50": 0.05015045135406219, "Perc90": 0.09027081243731194, "Perc95": 0.09528585757271817, "Perc99": 0.09929789368104314 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2", "extension_point": "PreScore" } }, { "data": { "Average": 0.341897013, "Perc50": 0.30685224839400427, "Perc90": 0.39250535331905784, "Perc95": 0.5052631578947369, "Perc99": 0.7859649122807019 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/ExtenderOnly/500Nodes/namespace-2", "extension_point": "Score" } }, { "data": { "Average": 934, "Perc50": 934, "Perc90": 934, "Perc95": 934, "Perc99": 934 }, "unit": "pods/s", "labels": { "Metric": "SchedulingThroughput", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2" } }, { "data": { "Average": 0.01762891699999999, "Perc50": 0.050200803212851405, "Perc90": 0.09036144578313253, "Perc95": 0.09538152610441766, "Perc99": 0.09939759036144578 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2", "extension_point": "PreFilter" } }, { "data": { "Average": 0.36928472399999995, "Perc50": 0.17127329192546584, "Perc90": 0.384549356223176, "Perc95": 0.593939393939394, "Perc99": 1.3333333333333335 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2", "extension_point": "Filter" } }, { "data": { "Average": 0.03252317899999998, "Perc50": 0.05005005005005006, "Perc90": 0.09009009009009009, "Perc95": 0.0950950950950951, "Perc99": 0.0990990990990991 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2", "extension_point": "PreScore" } }, { "data": { "Average": 0.49580472600000053, "Perc50": 0.33262599469496024, "Perc90": 0.6994871794871795, "Perc95": 0.8200000000000001, "Perc99": 1.9200000000000002 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2", "extension_point": "Score" } }, { "data": { "Average": 5.247761858000002, "Perc50": 2.6929133858267713, "Perc90": 11.25925925925926, "Perc95": 24.47058823529412, "Perc99": 55.27272727272728 }, "unit": "ms", "labels": { "Metric": "scheduler_scheduling_attempt_duration_seconds", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2" } }, { "data": { "Average": 347.6368404429999, "Perc50": 397.2078907435508, "Perc90": 591.4415781487102, "Perc95": 615.720789074355, "Perc99": 635.144157814871 }, "unit": "ms", "labels": { "Metric": "scheduler_pod_scheduling_duration_seconds", "Name": "BenchmarkPerfScheduling/PluginOnly/500Nodes/namespace-2" } }, { "data": { "Average": 110.55555555555556, "Perc50": 113, "Perc90": 116, "Perc95": 116, "Perc99": 116 }, "unit": "pods/s", "labels": { "Metric": "SchedulingThroughput", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2" } }, { "data": { "Average": 10.561972900000002, "Perc50": 12.184100418410042, "Perc90": 15.531380753138075, "Perc95": 15.949790794979082, "Perc99": 29.6 }, "unit": "ms", "labels": { "Metric": "scheduler_scheduling_attempt_duration_seconds", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2" } }, { "data": { "Average": 4531.955737822002, "Perc50": 4526.622516556292, "Perc90": 9049.302325581395, "Perc95": 9644.651162790698, "Perc99": 10120.93023255814 }, "unit": "ms", "labels": { "Metric": "scheduler_pod_scheduling_duration_seconds", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2" } }, { "data": { "Average": 0.020470835, "Perc50": 0.050200803212851405, "Perc90": 0.09036144578313253, "Perc95": 0.09538152610441766, "Perc99": 0.09939759036144578 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2", "extension_point": "PreFilter" } }, { "data": { "Average": 0.17684360699999965, "Perc50": 0.16203473945409427, "Perc90": 0.30329670329670333, "Perc95": 0.35824175824175825, "Perc99": 0.5 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2", "extension_point": "Filter" } }, { "data": { "Average": 0.09248618100000003, "Perc50": 0.05611672278338945, "Perc90": 0.11304347826086956, "Perc95": 0.1855072463768116, "Perc99": 0.7066666666666667 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2", "extension_point": "PreScore" } }, { "data": { "Average": 0.9478343950000003, "Perc50": 0.9684210526315791, "Perc90": 1.4947368421052631, "Perc95": 1.5605263157894738, "Perc99": 2.5411764705882356 }, "unit": "ms", "labels": { "Metric": "scheduler_framework_extension_point_duration_seconds", "Name": "BenchmarkPerfScheduling/AllEnabled/500Nodes/namespace-2", "extension_point": "Score" } } ] } ```

Which issue(s) this PR fixes:

Fixes #29

Special notes for your reviewer:

Does this PR introduce a user-facing change?

NONE

What are the benchmark results of this change?

n/a

sanposhiho commented 9 months ago

/kind feature

codefromthecrypt commented 9 months ago

great to see results validated!

/lgtm

k8s-ci-robot commented 9 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: codefromthecrypt, sanposhiho

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/kube-scheduler-wasm-extension/blob/main/OWNERS)~~ [codefromthecrypt,sanposhiho] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

kubernetes-sigs / kube-scheduler-wasm-extension