draios / sysdig

Linux system exploration and troubleshooting tool with first class support for containers
http://www.sysdig.com/
Other
7.78k stars 727 forks source link

Sysdig can't use k8s.pod.name as an output format #1756

Closed tzachiabo closed 11 months ago

tzachiabo commented 3 years ago

Hi,

I am trying to monitor syscalls with kubernetes context but when running this line of code I am not getting any response.

sudo sysdig  -k https://localhost:6443 -K /etc/kubernetes/pki/apiserver.crt:/etc/kubernetes/pki/apiserver.key -p"{\"pod_name\": \"%k8s.pod.name\"}"

What am I doing wrong? Do you have a guide regarding how to work with Kubernetes with Sysdig?

Thanks

nyrahul commented 3 years ago

This seems to be a bug. I tried the following:

  1. Used k8s.pod.name as the filter to check if I can get events in the context of the given pod name. The output was blank. Tried namespace and few other things with the same result.
  2. Tried csysdig and went to k8s pods view and it showed "Collecting data" forever.
  3. Then I tried sudo sysdig -pk -k proc.name=ls ... This will show events in k8s format for all the ls processes. Then I invoked ls process from one of the pod. It showed me following:
...
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":">","evt.info":"fd=1(<f>/dev/pts/0) size=125 ","evt.num":252853,"evt.outputtime":1620668960065975344,"evt.type":"write","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":"<","evt.info":"res=125 data=bin  boot  default.json  dev  etc  home  lib  lib64  media  mnt  opt  proc  root ","evt.num":252864,"evt.outputtime":1620668960066006212,"evt.type":"write","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":">","evt.info":"fd=1(<f>/dev/pts/0) ","evt.num":252872,"evt.outputtime":1620668960066013917,"evt.type":"close","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":"<","evt.info":"res=0 ","evt.num":252873,"evt.outputtime":1620668960066014788,"evt.type":"close","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":">","evt.info":"fd=2(<f>/dev/pts/0) ","evt.num":252874,"evt.outputtime":1620668960066015961,"evt.type":"close","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":"<","evt.info":"res=0 ","evt.num":252875,"evt.outputtime":1620668960066016391,"evt.type":"close","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":">","evt.info":"","evt.num":252884,"evt.outputtime":1620668960066033744,"evt.type":"exit_group","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
{"container.id":"23fc79186190","evt.cpu":1,"evt.dir":">","evt.info":"status=0 ","evt.num":252912,"evt.outputtime":1620668960066137609,"evt.type":"procexit","k8s.pod.name":null,"proc.name":"ls","thread.tid":21389,"thread.vtid":1049}
...

Note that the k8s.pod.name was null in all the cases. Thus the k8s context was printed but the pod name was clearly not working.

Note:

  1. kubectl output works fine on the same node.
  2. I was using a VM node setup using kubeadm init.
github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

therealbobo commented 11 months ago

Hi there! Sadly from the next release 0.35.0, since libs dropped the k8s client, we'll no longer support this feature. If you want to use feature you'll need a plugin.