kubernetes-sigs / kube-scheduler-wasm-extension

All the things to make the scheduler extendable with wasm.
Apache License 2.0
86 stars 17 forks source link

Use wire-type compatible source in TinyGo #17

Closed codefromthecrypt closed 11 months ago

codefromthecrypt commented 11 months ago

What type of PR is this?

/kind feature

What this PR does / why we need it:

Before, we had incompatible types, which among other things, made the POC difficult to reason with from a performance point of view. We were manually converting fields, which would never work in practice as the model is quite large.

This uses the real protos k8s does, just generating TinyGo compatible guest code from them. This now serializes the v1.Pod directly from the input type in a way TinyGo can get it. To eliminate a performance hit from filters that don't access the current pod, this extracts a FilterArgs type. Doing so gives us flexibility in implementation, especially as we add more parameters later.

Which issue(s) this PR fixes:

Fixes #13

Special notes for your reviewer:

This change slows down execution, but that's good because it is more realistic. Before we manually copied a single field, so the after is a better baseline. Note: this still doesn't use realistic data, but at least we can generate it now!

$ cd plugin
$ go test -run='^$' -bench '^BenchmarkFilter$' . -count=6 > old.txt
$ go test -run='^$' -bench '^BenchmarkFilter$' . -count=6 > new.txt
$ benchstat old.txt new.txt
goos: darwin
goarch: arm64
pkg: sigs.k8s.io/kube-scheduler-wasm-extension/plugin
                                                    │   old.txt   │              new.txt               │
                                                    │   sec/op    │   sec/op     vs base               │
Filter/success:_node_is_match_with_spec.NodeName-12   12.62µ ± 0%   19.14µ ± 1%  +51.73% (p=0.002 n=6)
codefromthecrypt commented 11 months ago

ps here is the output from the makefile. Note there are warnings about schema not being in use, but as they are _ imported you would have to change the source to exclude it.

$ make update-kubernetes-proto
cd ./kubernetes/proto/tools; \
    cat tools.go | grep "_" | awk -F'"' '{print $2}' | xargs -tI % go install %
go install github.com/knqyf263/go-plugin/cmd/protoc-gen-go-plugin
go install google.golang.org/protobuf/cmd/protoc-gen-go
echo "Regenerate the Go protobuf code."
Regenerate the Go protobuf code.
cd kubernetes/kubernetes/staging/src/; \
    protoc ./k8s.io/apimachinery/pkg/api/resource/generated.proto --go-plugin_out=../../../proto \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/api/resource/generated.proto=./resource; \
    protoc ./k8s.io/apimachinery/pkg/runtime/generated.proto --go-plugin_out=../../../proto \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/runtime/generated.proto=./runtime; \
    protoc ./k8s.io/apimachinery/pkg/runtime/schema/generated.proto --go-plugin_out=../../../proto \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/runtime/schema/generated.proto=./schema; \
    protoc ./k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto --go-plugin_out=../../../proto \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/apis/meta/v1/generated.proto=./meta \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/runtime/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/runtime \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/runtime/schema/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/schema; \
    protoc ./k8s.io/apimachinery/pkg/util/intstr/generated.proto --go-plugin_out=../../../proto \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/util/intstr/generated.proto=./instr; \
    protoc ./k8s.io/api/core/v1/generated.proto --go-plugin_out=../../../proto \
        --go-plugin_opt=Mk8s.io/api/core/v1/generated.proto=./api \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/api/resource/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/resource \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/apis/meta/v1/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/meta \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/runtime/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/runtime \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/runtime/schema/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/schema \
        --go-plugin_opt=Mk8s.io/apimachinery/pkg/util/intstr/generated.proto=sigs.k8s.io/kube-scheduler-wasm-extension/kubernetes/proto/instr;
k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/core/v1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s-ci-robot commented 11 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: codefromthecrypt, sanposhiho

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes-sigs/kube-scheduler-wasm-extension/blob/main/OWNERS)~~ [sanposhiho] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
kerthcet commented 11 months ago

Please squash you commit it said you have invalid commit message.

codefromthecrypt commented 11 months ago

ok I tried a different commit message 🤞

kerthcet commented 11 months ago

/lgtm

codefromthecrypt commented 11 months ago

Thanks for the fast feedback. now we are unlocked to do more modeling and performance work!