kmesh-net / kmesh

High Performance ServiceMesh Data Plane Based on Programmable Kernel
https://kmesh.net
Apache License 2.0
362 stars 46 forks source link

Kmesh panic when `curl 127.0.0.1:15200/bpf/kmesh/maps` #348

Closed hzxuzhonghu closed 1 month ago

hzxuzhonghu commented 1 month ago

Kmesh panicked when execute debug by request the status server.

How to reproduce:

run kmesh in workload mode, and then curl 127.0.0.1:15200/bpf/kmesh/maps in kmesh daemon.

More worse is that when after it panicked, it showed its still healthy

root@kurator-linux-0001:~# k get pod -n kmesh-system
NAME           READY   STATUS    RESTARTS   AGE
kmesh2-8nrwk   1/1     Running   0          11d
hzxuzhonghu commented 1 month ago

··· 2024-05-21T09:35:33.118654Z info http: panic serving 127.0.0.1:52042: runtime error: invalid memory address or nil pointer dereference goroutine 68388 [running]: net/http.(conn).serve.func1() /usr/local/go/src/net/http/server.go:1898 +0xbe panic({0x28736e0?, 0x4d54ba0?}) /usr/local/go/src/runtime/panic.go:770 +0x132 kmesh.net/kmesh/pkg/controller/dump.(StatusServer).httpBpfKmeshMaps(0x627b00?, {0x325eee0, 0xc00084ae00}, 0x7efeac823108?) /kmesh/pkg/controller/dump/status_server.go:97 +0x3f net/http.HandlerFunc.ServeHTTP(0xc0009180d0?, {0x325eee0?, 0xc00084ae00?}, 0x83aefa?) /usr/local/go/src/net/http/server.go:2166 +0x29 net/http.(ServeMux).ServeHTTP(0x46f059?, {0x325eee0, 0xc00084ae00}, 0xc000650900) /usr/local/go/src/net/http/server.go:2683 +0x1ad net/http.serverHandler.ServeHTTP({0xc000782180?}, {0x325eee0?, 0xc00084ae00?}, 0x6?) /usr/local/go/src/net/http/server.go:3137 +0x8e net/http.(conn).serve(0xc000afe120, {0x3270f38, 0xc000885350}) /usr/local/go/src/net/http/server.go:2039 +0x5e8 created by net/http.(Server).Serve in goroutine 29 /usr/local/go/src/net/http/server.go:3285 +0x4b4 2024-05-21T09:36:52.648387Z info http: panic serving 127.0.0.1:41496: runtime error: invalid memory address or nil pointer dereference goroutine 68392 [running]: net/http.(conn).serve.func1() /usr/local/go/src/net/http/server.go:1898 +0xbe panic({0x28736e0?, 0x4d54ba0?}) /usr/local/go/src/runtime/panic.go:770 +0x132 kmesh.net/kmesh/pkg/controller/dump.(StatusServer).httpBpfKmeshMaps(0x627b00?, {0x325eee0, 0xc00084aee0}, 0x7efeac823108?) /kmesh/pkg/controller/dump/status_server.go:97 +0x3f net/http.HandlerFunc.ServeHTTP(0xc0009180d0?, {0x325eee0?, 0xc00084aee0?}, 0x83aefa?) /usr/local/go/src/net/http/server.go:2166 +0x29 net/http.(ServeMux).ServeHTTP(0x46f059?, {0x325eee0, 0xc00084aee0}, 0xc000650a20) /usr/local/go/src/net/http/server.go:2683 +0x1ad net/http.serverHandler.ServeHTTP({0xc000782a50?}, {0x325eee0?, 0xc00084aee0?}, 0x6?) /usr/local/go/src/net/http/server.go:3137 +0x8e net/http.(conn).serve(0xc000aff200, {0x3270f38, 0xc000885350}) /usr/local/go/src/net/http/server.go:2039 +0x5e8 created by net/http.(Server).Serve in goroutine 29 /usr/local/go/src/net/http/server.go:3285 +0x4b4

···