search

k3s-io / k3s

Lightweight Kubernetes
https://k3s.io
Apache License 2.0
27.94k stars 2.33k forks source link

[Release-1.30] - pprof statistics cannot be access when running server with `--disable-agent` #10633

Closed brandond closed 2 months ago

brandond commented 2 months ago

Backport fix for pprof statistics cannot be access when running server with --disable-agent

ShylajaDevadiga commented 2 months ago

Validated using k3s commit id on release-1.30 branch v1.30.3+k3s-a125b7f6

Environment Details

Infrastructure Cloud EC2 instance

Node(s) CPU architecture, OS, and Version:

> cat /etc/os-release
NAME="SLES"
VERSION="15-SP5"
VERSION_ID="15.5"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration: Single node

Steps to reproduce

  1. Install k3s passing --disable-agent --enable-pprof
  2. Open kube proxy running kubectl proxy
  3. Capture the profile data using go tool pprof --seconds=70 -top http://localhost:8001/debug/pprof/profile

Replication results:

ec2-user@ip-172-31-6-0:~> k3s -v
k3s version v1.30.3+k3s1 (f6466040)
go version go1.22.5
ec2-user@ip-172-31-6-0:~> /usr/local/go/bin/go tool pprof --seconds=70 -top http://localhost:8001/debug/pprof/profile 
Fetching profile over HTTP from http://localhost:8001/debug/pprof/profile?seconds=70
Please wait... (1m10s)
http://localhost:8001/debug/pprof/profile: server response: 404 Not Found
failed to fetch any source profiles
ec2-user@ip-172-31-6-0:~>

Validation results:

> k3s -v
k3s version v1.30.3+k3s-a125b7f6 (a125b7f6)
go version go1.22.5

> /usr/local/go/bin/go tool pprof --seconds=70 -top http://localhost:8001/debug/pprof/profile 
Fetching profile over HTTP from http://localhost:8001/debug/pprof/profile?seconds=70
Please wait... (1m10s)
Saved profile in /home/ec2-user/pprof/pprof.k3s.samples.cpu.001.pb.gz
File: k3s
Build ID: dfd0dab14a5377388dc9c2571cb2fe1d8851ca18
Type: cpu
Time: Aug 13, 2024 at 5:34pm (UTC)
Duration: 70s, Total samples = 1.65s ( 2.36%)
Showing nodes accounting for 1.65s, 100% of 1.65s total
      flat  flat%   sum%        cum   cum%
     0.34s 20.61% 20.61%      0.34s 20.61%  runtime/internal/syscall.Syscall6
     0.25s 15.15% 35.76%      0.25s 15.15%  runtime.futex
     0.08s  4.85% 40.61%      0.08s  4.85%  runtime.cgocall
     0.08s  4.85% 45.45%      0.08s  4.85%  runtime.write1
     0.06s  3.64% 49.09%      0.10s  6.06%  runtime.mallocgc
     0.06s  3.64% 52.73%      0.06s  3.64%  runtime.siftdownTimer