search

kubeslice / worker-operator

Kubeslice Worker Operator Opensource Repository: The KubeSlice Worker Operator is a Kubernetes operator that manages the lifecycle of KubeSlice worker clusters.
Apache License 2.0
58 stars 19 forks source link

Bug: Operator panicking out as the `decoder` is not set #338

Closed Bhargav-InfraCloud closed 4 months ago

Bhargav-InfraCloud commented 4 months ago

📜 Description

After the slice is all set between worker clusters when trying to create an application deployment (Eg. iperf app), there is a webhook error:

Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.kubeslice.io": Post "[https://kubeslice-webhook-service.kubeslice-system.svc:443/mutate-webhook?timeout=10s](https://kubeslice-webhook-service.kubeslice-system.svc/mutate-webhook?timeout=10s)": EOF

Upon debugging, we've found that this is caused by the same sigs.k8s.io/controller-runtime's version update as explained in #335, but for a different injection method: InjectDecoder.

CC: @soharab-ic

👟 Reproduction steps

  1. Run the controller and worker setup.
  2. Create a deployment in the namespace. You may use this deployment: https://kubeslice.io/documentation/open-source/0.2.0/getting-started-with-kind-clusters/#deploying-the-iperf-application It throws webhook error while doing a kubectl apply...: 
    Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "webhook.kubeslice.io": Post "[https://kubeslice-webhook-service.kubeslice-system.svc:443/mutate-webhook?timeout=10s](https://kubeslice-webhook-service.kubeslice-system.svc/mutate-webhook?timeout=10s)": EOF
  3. Check for the kubeslice-operator deployment's logs. There should be a nil pointer deference (attached the logs below).

👍 Expected behavior

It shouldn't throw a nil pointer dereference error. And allow creation of resources (like deployment) in the associated namespace.

👎 Actual Behavior

It throws a nil pointer dereference error. And doesn't allow the creation of resources (like deployment) in the associated namespace.

🐚 Relevant log output

{"severity":"info","time":"2024-02-28T07:41:32Z","logger":"controllers.Slice","message":"ExternalGatewayConfig","namespace":"kubeslice-system","sliceCluster":"kind-kubeslice-worker-2","slice":{"name":"water","namespace":"kubeslice-system"},"egw":{"sliceId":"","sliceDisplayName":"water","sliceSubnet":"10.1.0.0/16","sliceType":"Application","qosProfileDetails":{"queueType":"HTB","bandwidthCeilingKbps":5120,"bandwidthGuaranteedKbps":2560,"priority":1,"dscpClass":"AF11","tcType":"BANDWIDTH_CONTROL"},"sliceIpam":{"sliceIpamType":"Local"},"externalGatewayConfig":{"ingress":{},"egress":{},"nsIngress":{}},"namespaceIsolationProfile":{"applicationNamespaces":["iperf"],"allowedNamespaces":["kube-system"]},"clusterSubnetCIDR":"10.1.16.0/20","sliceGatewayServiceType":"NodePort","sliceGatewayProtocol":"UDP"}}
2024/02/28 07:41:35 http: panic serving 172.18.0.7:7904: runtime error: invalid memory address or nil pointer dereference
goroutine 14245 [running]:
net/http.(*conn).serve.func1()
    /usr/local/go/src/net/http/server.go:1854 +0xbf
panic({0x1a0e860, 0x2caf3a0})
    /usr/local/go/src/runtime/panic.go:890 +0x263
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Decoder).DecodeRaw(0x0, {{0xc000bc8000, 0xc25, 0xc80}, {0x0, 0x0}}, {0x1f11180, 0xc0002b0000})
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/decode.go:77 +0x15a
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Decoder).Decode(_, {{{0xc0002ed6e0, 0x24}, {{0xc00116d100, 0x4}, {0xc00116d104, 0x2}, {0xc00116d106, 0xa}}, {{0xc00116d110, ...}, ...}, ...}}, ...)
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/decode.go:49 +0x8e
github.com/kubeslice/worker-operator/pkg/webhook/pod.(*WebhookServer).Handle(_, {_, _}, {{{0xc0002ed6e0, 0x24}, {{0xc00116d100, 0x4}, {0xc00116d104, 0x2}, {0xc00116d106, ...}}, ...}})
    /workspace/pkg/webhook/pod/webhook.go:93 +0x78a
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).Handle(_, {_, _}, {{{0xc0002ed6e0, 0x24}, {{0xc00116d100, 0x4}, {0xc00116d104, 0x2}, {0xc00116d106, ...}}, ...}})
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/webhook.go:169 +0x20a
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP(0xc000614230, {0x7feae6a3ea98?, 0xc00113d220}, 0xc000015900)
    /workspace/vendor/sigs.k8s.io/controller-runtime/pkg/webhook/admission/http.go:98 +0xc94
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerInFlight.func1({0x7feae6a3ea98, 0xc00113d220}, 0x1f23a00?)
    /workspace/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:60 +0xd4
net/http.HandlerFunc.ServeHTTP(0x1f23a40?, {0x7feae6a3ea98?, 0xc00113d220?}, 0xc0007f7828?)
    /usr/local/go/src/net/http/server.go:2122 +0x2f
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerCounter.func1({0x1f23a40?, 0xc000b0a700?}, 0xc000015900)
    /workspace/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:147 +0xc5
net/http.HandlerFunc.ServeHTTP(0x7d1e05?, {0x1f23a40?, 0xc000b0a700?}, 0x7feb0e366108?)
    /usr/local/go/src/net/http/server.go:2122 +0x2f
github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2({0x1f23a40, 0xc000b0a700}, 0xc000015900)
    /workspace/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:109 +0xc7
net/http.HandlerFunc.ServeHTTP(0xc000b0a700?, {0x1f23a40?, 0xc000b0a700?}, 0x1c69517?)
    /usr/local/go/src/net/http/server.go:2122 +0x2f
net/http.(*ServeMux).ServeHTTP(0xc0002ed6c5?, {0x1f23a40, 0xc000b0a700}, 0xc000015900)
    /usr/local/go/src/net/http/server.go:2500 +0x149
net/http.serverHandler.ServeHTTP({0x1f14178?}, {0x1f23a40, 0xc000b0a700}, 0xc000015900)
    /usr/local/go/src/net/http/server.go:2936 +0x316
net/http.(*conn).serve(0xc000de3170, {0x1f24848, 0xc00049e2d0})
    /usr/local/go/src/net/http/server.go:1995 +0x612
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:3089 +0x5ed
{"severity":"info","time":"2024-02-28T07:41:38Z","logger":"controllers.SliceGw","message":"reconciling","namespace":"kubeslice-system","sliceCluster":"kind-kubeslice-worker-2","slicegateway":{"name":"water-kind-kubeslice-worker-2-kind-kubeslice-worker-1","namespace":"kubeslice-system"},"slice":"water","slicegateway":"water-kind-kubeslice-worker-2-kind-kubeslice-worker-1","NumberOfGateways":2}

Version

Latest master. Commit: b307957037d7e589e27fc820a8bc310e625e0e00

🖥️ What operating system are you seeing the problem on?

Linux

✅ Proposed Solution

Set the decoder field in the main package itself, while initializing WebhookServer.

👀 Have you spent some time to check if this issue has been raised before?

Code of Conduct