search

falcosecurity / falco

Cloud Native Runtime Security
https://falco.org
Apache License 2.0
7.24k stars 893 forks source link

Falco pods CrashLoopBackOff in Minikube Cluster #3155

Closed m-muzammil786 closed 28 minutes ago

m-muzammil786 commented 5 months ago

When I run falco in aws ubuntu machine using minikube cluster this show error, I am using helm chart, ubuntu@ip-172-31-42-24:~$ uname -r 6.5.0-1016-aws ubuntu@ip-172-31-42-24:~$ ubuntu@ip-172-31-42-24:~$ helm repo list NAME URL falcosecurity https://falcosecurity.github.io/charts ubuntu@ip-172-31-42-24:~$ helm install falco falcosecurity/falco NAME: falco LAST DEPLOYED: Fri Apr 5 11:31:13 2024 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None NOTES: Falco agents are spinning up on each node in your cluster. After a few seconds, they are going to start monitoring your containers looking for security issues.

No further action should be required.

Tip: You can easily forward Falco events to Slack, Kafka, AWS Lambda and more with falcosidekick. Full list of outputs: https://github.com/falcosecurity/charts/tree/master/charts/falcosidekick. You can enable its deployment with --set falcosidekick.enabled=true or in your values.yaml. See: https://github.com/falcosecurity/charts/blob/master/charts/falcosidekick/values.yaml for configuration values. ubuntu@ip-172-31-42-24:~$ kubectl get pods -w NAME READY STATUS RESTARTS AGE falco-s6wg8 0/2 Init:0/2 0 15s falco-s6wg8 0/2 Init:0/2 0 28s falco-s6wg8 0/2 Init:1/2 0 29s falco-s6wg8 0/2 Init:1/2 0 32s falco-s6wg8 0/2 PodInitializing 0 33s falco-s6wg8 1/2 Error 0 40s falco-s6wg8 1/2 Error 1 (1s ago) 41s falco-s6wg8 1/2 CrashLoopBackOff 1 (1s ago) 42s falco-s6wg8 1/2 Error 2 (20s ago) 61s ^Cubuntu@ip-172-31-42-24:~$ kd falco-s6wg8 error: the server doesn't have a resource type "falco-s6wg8" ubuntu@ip-172-31-42-24:~$ kd pods falco-s6wg8 Name: falco-s6wg8 Namespace: default Priority: 0 Service Account: default Node: minikube/192.168.49.2 Start Time: Fri, 05 Apr 2024 11:31:13 +0000 Labels: app.kubernetes.io/instance=falco app.kubernetes.io/name=falco controller-revision-hash=9fd8499fd pod-template-generation=1 Annotations: checksum/certs: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 checksum/config: c7580c3802ee5537b2aa31e3e4dde1d9afecb4ea70f9c86c3952a7d44cd59cf0 checksum/rules: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 Status: Running IP: 10.244.0.6 IPs: IP: 10.244.0.6 Controlled By: DaemonSet/falco Init Containers: falco-driver-loader: Container ID: docker://a7c23f98b4a05428e2267819b6141738198ffffcc44a0464b90943935243b8c1 Image: docker.io/falcosecurity/falco-driver-loader:0.37.1 Image ID: docker-pullable://falcosecurity/falco-driver-loader@sha256:e1389978dbee6c55c4f712f9f43d875e761578cb828965f33402c4fe14351df1 Port: Host Port: State: Terminated Reason: Completed Exit Code: 0 Started: Fri, 05 Apr 2024 11:31:40 +0000 Finished: Fri, 05 Apr 2024 11:31:41 +0000 Ready: True Restart Count: 0 Environment: Mounts: /host/boot from boot-fs (ro) /host/etc from etc-fs (ro) /host/lib/modules from lib-modules (rw) /host/proc from proc-fs (ro) /host/usr from usr-fs (ro) /root/.falco from root-falco-fs (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-swsf8 (ro) falcoctl-artifact-install: Container ID: docker://98d61f4eaa4f6e8a9b838398436fbaca80f4032ed466bda301f19de5b404224c Image: docker.io/falcosecurity/falcoctl:0.7.2 Image ID: docker-pullable://falcosecurity/falcoctl@sha256:6b4f448f82fc7e12d4ce27213cbcc8eaa47bef28f78817b77d027ef12801c984 Port: Host Port: Args: artifact install --log-format=json State: Terminated Reason: Completed Exit Code: 0 Started: Fri, 05 Apr 2024 11:31:44 +0000 Finished: Fri, 05 Apr 2024 11:31:45 +0000 Ready: True Restart Count: 0 Environment: Mounts: /etc/falcoctl from falcoctl-config-volume (rw) /plugins from plugins-install-dir (rw) /rulesfiles from rulesfiles-install-dir (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-swsf8 (ro) Containers: falco: Container ID: docker://e06ae29aa93e0fe5366657d8a43175c358a9bb08a1493b1ade8d8b8e6ee9b17c Image: docker.io/falcosecurity/falco-no-driver:0.37.1 Image ID: docker-pullable://falcosecurity/falco-no-driver@sha256:391c4bfd42331d1f1909d19827dcf4aa7ba7bb7984066aefc1c14cc4f04c0775 Port: Host Port: Args: /usr/bin/falco --cri /run/containerd/containerd.sock --cri /run/crio/crio.sock -pk State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 Started: Fri, 05 Apr 2024 11:32:14 +0000 Finished: Fri, 05 Apr 2024 11:32:14 +0000 Ready: False Restart Count: 2 Limits: cpu: 1 memory: 1Gi Requests: cpu: 100m memory: 512Mi Liveness: http-get http://:8765/healthz delay=60s timeout=5s period=15s #success=1 #failure=3 Readiness: http-get http://:8765/healthz delay=30s timeout=5s period=15s #success=1 #failure=3 Environment: FALCO_K8S_NODE_NAME: (v1:spec.nodeName) Mounts: /etc/falco from rulesfiles-install-dir (rw) /etc/falco/falco.yaml from falco-yaml (rw,path="falco.yaml") /host/dev from dev-fs (ro) /host/etc from etc-fs (ro) /host/proc from proc-fs (rw) /host/run/containerd/containerd.sock from containerd-socket (rw) /host/run/crio/crio.sock from crio-socket (rw) /host/var/run/docker.sock from docker-socket (rw) /root/.falco from root-falco-fs (rw) /sys/module/falco from sys-fs (rw) /usr/share/falco/plugins from plugins-install-dir (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-swsf8 (ro) falcoctl-artifact-follow: Container ID: docker://0b6d180db2b18b67b5fc8cd0621af776ff8d82c11c66e24980d97e88071f8eed Image: docker.io/falcosecurity/falcoctl:0.7.2 Image ID: docker-pullable://falcosecurity/falcoctl@sha256:6b4f448f82fc7e12d4ce27213cbcc8eaa47bef28f78817b77d027ef12801c984 Port: Host Port: Args: artifact follow --log-format=json State: Running Started: Fri, 05 Apr 2024 11:31:53 +0000 Ready: True Restart Count: 0 Environment: Mounts: /etc/falcoctl from falcoctl-config-volume (rw) /plugins from plugins-install-dir (rw) /rulesfiles from rulesfiles-install-dir (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-swsf8 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: plugins-install-dir: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: rulesfiles-install-dir: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: root-falco-fs: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: boot-fs: Type: HostPath (bare host directory volume) Path: /boot HostPathType: lib-modules: Type: HostPath (bare host directory volume) Path: /lib/modules HostPathType: usr-fs: Type: HostPath (bare host directory volume) Path: /usr HostPathType: etc-fs: Type: HostPath (bare host directory volume) Path: /etc HostPathType: dev-fs: Type: HostPath (bare host directory volume) Path: /dev HostPathType: sys-fs: Type: HostPath (bare host directory volume) Path: /sys/module/falco HostPathType: docker-socket: Type: HostPath (bare host directory volume) Path: /var/run/docker.sock HostPathType: containerd-socket: Type: HostPath (bare host directory volume) Path: /run/containerd/containerd.sock HostPathType: crio-socket: Type: HostPath (bare host directory volume) Path: /run/crio/crio.sock HostPathType: proc-fs: Type: HostPath (bare host directory volume) Path: /proc HostPathType: falcoctl-config-volume: Type: ConfigMap (a volume populated by a ConfigMap) Name: falco-falcoctl Optional: false falco-yaml: Type: ConfigMap (a volume populated by a ConfigMap) Name: falco Optional: false kube-api-access-swsf8: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: DownwardAPI: true QoS Class: Burstable Node-Selectors: Tolerations: node-role.kubernetes.io/control-plane:NoSchedule node-role.kubernetes.io/master:NoSchedule node.kubernetes.io/disk-pressure:NoSchedule op=Exists node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists node.kubernetes.io/pid-pressure:NoSchedule op=Exists node.kubernetes.io/unreachable:NoExecute op=Exists node.kubernetes.io/unschedulable:NoSchedule op=Exists Events: Type Reason Age From Message

Normal Scheduled 88s default-scheduler Successfully assigned default/falco-s6wg8 to minikube Normal Pulling 87s kubelet Pulling image "docker.io/falcosecurity/falco-driver-loader:0.37.1" Normal Pulled 61s kubelet Successfully pulled image "docker.io/falcosecurity/falco-driver-loader:0.37.1" in 25.822s (25.822s including waiting) Normal Created 61s kubelet Created container falco-driver-loader Normal Started 61s kubelet Started container falco-driver-loader Normal Pulling 59s kubelet Pulling image "docker.io/falcosecurity/falcoctl:0.7.2" Normal Started 57s kubelet Started container falcoctl-artifact-install Normal Created 57s kubelet Created container falcoctl-artifact-install Normal Pulled 57s kubelet Successfully pulled image "docker.io/falcosecurity/falcoctl:0.7.2" in 2.082s (2.082s including waiting) Normal Pulling 55s kubelet Pulling image "docker.io/falcosecurity/falco-no-driver:0.37.1" Normal Pulled 49s kubelet Successfully pulled image "docker.io/falcosecurity/falco-no-driver:0.37.1" in 5.647s (5.647s including waiting) Normal Pulled 49s kubelet Container image "docker.io/falcosecurity/falcoctl:0.7.2" already present on machine Normal Created 49s kubelet Created container falcoctl-artifact-follow Normal Started 48s kubelet Started container falcoctl-artifact-follow Normal Created 27s (x3 over 49s) kubelet Created container falco Normal Started 27s (x3 over 49s) kubelet Started container falco Normal Pulled 27s (x2 over 48s) kubelet Container image "docker.io/falcosecurity/falco-no-driver:0.37.1" already present on machine Warning BackOff 27s (x4 over 47s) kubelet Back-off restarting failed container falco in pod falco-s6wg8_default(4ca8a433-3707-49f4-b998-3fd6ed65c87c) ubuntu@ip-172-31-42-24:~$ kl falco-s6wg8 Defaulted container "falco" out of: falco, falcoctl-artifact-follow, falco-driver-loader (init), falcoctl-artifact-install (init) Fri Apr 5 11:32:43 2024: Falco version: 0.37.1 (x86_64) Fri Apr 5 11:32:43 2024: Falco initialized with configuration file: /etc/falco/falco.yaml Fri Apr 5 11:32:43 2024: System info: Linux version 6.5.0-1016-aws (buildd@lcy02-amd64-078) (x86_64-linux-gnu-gcc-11 (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #16~22.04.1-Ubuntu SMP Wed Mar 13 18:54:49 UTC 2024 Fri Apr 5 11:32:43 2024: Loading rules from file /etc/falco/falco_rules.yaml Fri Apr 5 11:32:43 2024: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs) Fri Apr 5 11:32:43 2024: Starting health webserver with threadiness 2, listening on 0.0.0.0:8765 Fri Apr 5 11:32:43 2024: Loaded event sources: syscall Fri Apr 5 11:32:43 2024: Enabled event sources: syscall Fri Apr 5 11:32:43 2024: Opening 'syscall' source with Kernel module Fri Apr 5 11:32:43 2024: Trying to inject the Kernel module and opening the capture again... Fri Apr 5 11:32:43 2024: Unable to load the driver Fri Apr 5 11:32:43 2024: An error occurred in an event source, forcing termination... Events detected: 0 Rule counts by severity: Triggered rules by rule name: Error: error opening device /host/dev/falco0. Make sure you have root credentials and that the falco module is loaded: No such file or directory ubuntu@ip-172-31-42-24:~$

plz tell me how to solve this error?

poiana commented 5 months ago

There is not a label identifying the kind of this issue. Please specify it either using /kind <group> or manually from the side menu.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
Andreagit97 commented 5 months ago

ei! you could try to use the modern_ebpf driver instead of the kmod (https://github.com/falcosecurity/charts/blob/91bfff2bf1127c4687f9e4bc4eaab68f77e5b91e/charts/falco/values.yaml#L177)

LucaGuerra commented 5 months ago

The getting started guide for minikube was just updated by @alacuku : https://falco.org/docs/install-operate/third-party/learning/ . Would you mind following the instructions there?

poiana commented 2 months ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana commented 1 month ago

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

poiana commented 28 minutes ago

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community. /close

poiana commented 28 minutes ago

@poiana: Closing this issue.

In response to [this](https://github.com/falcosecurity/falco/issues/3155#issuecomment-2350940106): >Rotten issues close after 30d of inactivity. > >Reopen the issue with `/reopen`. > >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Provide feedback via https://github.com/falcosecurity/community. >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.