search

eldadru / ksniff

Kubectl plugin to ease sniffing on kubernetes pods using tcpdump and wireshark
Apache License 2.0
3.13k stars 186 forks source link

K8S API client 30s timeout: request canceled (Client.Timeout exceeded while awaiting headers) #179

Open mgerasimchuk opened 3 months ago

mgerasimchuk commented 3 months ago

I try to sniff the traffic by using a privileged pod, but face k8s API timeout:

» ~  kubectl sniff -p -n default nginx-69f9fb6f84 --pod-creation-timeout 520s
INFO[0001] sniffing method: privileged pod
INFO[0001] sniffing on pod: 'nginx-69f9fb6f84' [namespace: 'default', filter: '', interface: 'any']
INFO[0001] creating privileged pod on node: 'ip-10-12-1-1.ec2.internal'
ERRO[0031] failed to create privileged pod on node: 'ip-10-12-1-1.ec2.internal'  error="Post \"https://k8s-cluster.us-east-1.eks.amazonaws.com/api/v1/namespaces/default/pods?timeout=30s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Error: Post "https://k8s-cluster.us-east-1.eks.amazonaws.com/api/v1/namespaces/default/pods?timeout=30s": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

For now, it's not possible to change timeout, cos it is hardcoded by 30s here: https://github.com/eldadru/ksniff/blob/b0b737c/pkg/cmd/sniff.go#L231

WORKAROUND (just use changes from the related PR):

For the workaround, you can build ksnif from PR sources and replace it.\ Example for the MacOS on ARM:

» ~  cd /tmp
» /tmp  git clone -b feature/request-timeout-flag git@github.com:mgerasimchuk/ksniff.git 
Cloning into 'ksniff'...
Resolving deltas: 100% (293/293), done.
» /tmp  cd ksniff 
» /tmp/ksniff (feature/request-timeout-flag) make darwin
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
GO111MODULE=on GOOS=darwin GOARCH=amd64 go build -o kubectl-sniff-darwin cmd/kubectl-sniff.go
GO111MODULE=on GOOS=darwin GOARCH=arm64 go build -o kubectl-sniff-darwin-arm64 cmd/kubectl-sniff.go
» /tmp/ksniff (feature/request-timeout-flag) cp kubectl-sniff-darwin-arm64 ~/.krew/bin/kubectl-sniff

» /tmp/ksniff (feature/request-timeout-flag) kubectl sniff --help | grep timeout
      --pod-creation-timeout duration   the length of time to wait for privileged pod to be created (e.g. 20s, 2m, 1h). A value of zero means the creation never times out. (default 1m0s)
      --request-timeout duration        the length of time to wait before giving up on a single request to the kubernetes api (e.g. 20s, 2m, 1h). A value of zero means the creation never times out. (default 30s)

» ~  kubectl sniff --request-timeout 520s --pod-creation-timeout 520s -p -n default nginx-69f9fb6f84