search

falcosecurity / falco

Cloud Native Runtime Security
https://falco.org
Apache License 2.0
7.24k stars 893 forks source link

falco pods are into Init:CrashLoopBackOff state #3306

Closed rshiva777 closed 5 days ago

rshiva777 commented 2 weeks ago

Describe the bug

Falco pods are not coming into running state. it throws an error stating module not found as below

"Trying to download a prebuilt falco module from https://download.falco.org/driver/6.0.1%2Bdriver/x86_64/falco_amazonlinux2_5.10.223-212.873.amzn2.x86_64_1.ko curl: (22) The requested URL returned error: 404"

If i try to download the ko file, its thowing 404, seems like it doesnt exist. even falco module build is failed with the error as below:

" LD [M] /var/lib/dkms/falco/6.0.1+driver/build/falco.o /bin/sh: 1: gcc10-ld.bfd: not found make[3]: [scripts/Makefile.build:437: /var/lib/dkms/falco/6.0.1+driver/build/falco.o] Error 127 make[2]: [Makefile.kernel:1829: /var/lib/dkms/falco/6.0.1+driver/build] Error 2 make[2]: Leaving directory '/host/usr/src/kernels/5.10.223-212.873.amzn2.x86_64' make[1]: [Makefile:9: modules] Error 2 make[1]: Leaving directory '/host/usr/src/kernels/5.10.223-212.873.amzn2.x86_64' make: [Makefile:16: all] Error 2

How to reproduce it

Install falco using helm chart version "0.36.2" OS: Amazon Linux Kernel Version: 5.10.223-212.873.amzn2.x86_64

Expected behaviour

Falco pods should be running without any issues

Screenshots

image image

Environment

[root@ip-10-0-1-24 ~]# cat /etc/os-release NAME="Amazon Linux" VERSION="2" ID="amzn" ID_LIKE="centos rhel fedora" VERSION_ID="2" PRETTY_NAME="Amazon Linux 2" ANSI_COLOR="0;33" CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2" HOME_URL="https://amazonlinux.com/" SUPPORT_END="2025-06-30" [root@ip-10-0-1-24 ~]# uname -a Linux ip-10-0-1-24.eu-west-1.compute.internal 5.10.223-212.873.amzn2.x86_64 #1 SMP Wed Aug 7 16:53:32 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux [root@ip-10-0-1-24 ~]#

Additional context

ashtongraves commented 2 weeks ago

Adding onto this, getting the same error when it's trying to pull https://download.falco.org/driver/7.2.1%2Bdriver/aarch64/falco_ubuntu-nvidia_6.2.0-1015-nvidia-64k_15.o

This driver type is ebpf and my os info is as follows:

PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

kernel: 6.2.0-1015-nvidia-64k

FedeDP commented 2 weeks ago

Hi @rshiva777 ! Your issue is that you are using a Falco version whose driver is now unsupported; we provide prebuilt artifacts for latest 3 driver versions, in this moment they are: [ 7.0.0+driver, 7.2.0+driver, 7.2.1+driver ]: https://github.com/falcosecurity/test-infra/tree/master/driverkit/config

Indeed, i can see the prebuilt available for eg: 7.2.1+driver:

2024-08-20T08:42:04.000Z 1.0 MB falco_amazonlinux2_5.10.223-212.873.amzn2.x86_64_1.ko

@ashtongraves we don't support prebuilt drivers for ubuntu-nvidia flavor; we only support: aws, azure, gcp, generic and gke flavors. See https://github.com/falcosecurity/test-infra/tree/master/config/jobs/build-drivers for the list of supported prebuilt drivers.

You'd both better rely on the modern_ebpf driver for these kernels anyway! It does not need any prebuilt artifact.

rshiva777 commented 2 weeks ago

@FedeDP ,

We are currently using falco helm chart: 4.6.0 appversion: 0.38.2

This is working as expected on operating system as below:

[root@ip-10-0-1-179 ~]# uname -r
5.10.223-212.873.amzn2.x86_64
[root@ip-10-0-1-179 ~]#
[root@ip-10-0-1-179 ~]# cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
SUPPORT_END="2025-06-30"
[root@ip-10-0-1-179 ~]# ^C
[root@ip-10-0-1-179 ~]#
image

If i try to use same kernel with falco helm chart: 3.8.7 appversion: 0.36.2

[root@ip-10-0-1-179 ~]# uname -r
5.10.223-212.873.amzn2.x86_64
[root@ip-10-0-1-179 ~]#
[root@ip-10-0-1-179 ~]# cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
SUPPORT_END="2025-06-30"
[root@ip-10-0-1-179 ~]# ^C
[root@ip-10-0-1-179 ~]#
image

So, falco drivers are bundled with falco docker images ?

what i can see is,

falco image 0.36.2 is using 6.0.1+driver falco image 0.38.0 is usin 7.2.0+driver

rshiva777 commented 2 weeks ago

how can i see which version of EKS nodes (kernel) are compatible with falco image or driver versions. i dont see in documentation

FedeDP commented 1 week ago

So, falco drivers are bundled with falco docker images ?

Yes, each Falco has its own driver version basically.

how can i see which version of EKS nodes (kernel) are compatible with falco image or driver versions

You can go here: https://download.falco.org/driver/site/index.html?lib=7.2.0%2Bdriver&target=amazonlinux2&arch=x86_64&kind=ebpf and look for your kernel for your desired driver version; if it is there, then your kernel is supported.

Also, bear in mind that we only support prebuild drivers for latest 3 driver versions, as previously said.

rshiva777 commented 1 week ago

Thanks for the info @FedeDP

Also, do we have any page which show the supported driver version along with the docker image for falco

FedeDP commented 1 week ago

Oh that's quite easy since some rleeases of Falco: it is right in the top of the github release page: https://github.com/falcosecurity/falco/releases/tag/0.38.2

Eg: immagine

FedeDP commented 5 days ago

/milestone 0.39.0 /close

poiana commented 5 days ago

@FedeDP: Closing this issue.

In response to [this](https://github.com/falcosecurity/falco/issues/3306#issuecomment-2337237768): >/milestone 0.39.0 >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository.