Falco crashLoopBack due to "Unable to find a prebuilt falco module" error

[hasakura12]

Describe the bug

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update

helm install falco \
    -f custom_rules.yaml \
    -f overrides.yaml \
    falcosecurity/falco

$ k get po
falco-5j5sd                                      0/1     CrashLoopBackOff    2          21s

$ k logs falco-5j5sd 
* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.31.1, driver version=b7eb0dd65226a8dc254d228c8d950d07bf3521d2
* Running falco-driver-loader with: driver=module, compile=yes, download=yes
* Unloading falco module, if present
* Looking for a falco module locally (kernel 5.4.214-120.368.amzn2.x86_64)
* Trying to download a prebuilt falco module from https://download.falco.org/driver/b7eb0dd65226a8dc254d228c8d950d07bf3521d2/falco_amazonlinux2_5.4.214-120.368.amzn2.x86_64_1.ko
curl: (22) The requested URL returned error: 404 
Unable to find a prebuilt falco module
* Trying to dkms install falco module with GCC /usr/bin/gcc

How to reproduce it

Expected behaviour

Should be running

Screenshots

Environment

• Falco version:
• System info:
• Cloud provider or hardware configuration:
• OS:
• Kernel:
• Installation method:

Additional context

[alacuku]

Hi @hasakura12, the chart is deploying Falco 0.31.1, is that intentional? If yes unfortunately the pre-built modules are not available for your kernel. You should install the kernel headers on the machines where you are installing Falco and the Falco pod will try to build the kernel module for you.

[alacuku]

Anyway you can check here if the prebuilt module exists for your kernel: https://download.falco.org/driver/site/index.html?lib=3.0.1%2Bdriver&target=amazonlinux2&arch=x86_64&kind=kmod

[hasakura12]

@alacuku

the chart is deploying Falco 0.31.1, is that intentional?

Not intentional. I just want to deploy Falco tbh, and I did it a few months ago and it was working until a few weeks back.

Do you suggest other working version instead?

[alacuku]

Do you suggest other working version instead?

Unfortunately, the pre-built module for amazonlinux2_5.4.214-120.368 is not available yet. You should install the kernel header in order to get falco to work. Otherwise, you can wait until the kernel-crawler finds the new kernel and our test-infra builds it.

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[poiana]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

[dcardellino]

Is there any update on this? Running also in this issue...

[alacuku]

Hi @dcardellino, kernel modules not being available for a given kernel version is not a chart's issue. You have three options:

1. Wait for a module to be available on https://download.falco.org/?prefix=driver/4.0.0%2Bdriver/. Keep in mind that the discovery and building of modules for new kernels are done in best-effort mode. Please see https://github.com/falcosecurity/kernel-crawler;
2. Build the falco module by your self and load it on each cluster node before deploying falco;
3. Install kernel headers on each node, and the falco-driver-loader will build the module before running falco.

[dcardellino]

@alacuku So I installed the kernel headers like you said but now I got some other weird error message 🤷‍♂️

falco-driver-loader warning: the compiler differs from the one used to build the kernel
falco-driver-loader   The kernel was built by: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
falco-driver-loader   You are using:           gcc-8 (Debian 8.3.0-6) 8.3.0
falco-driver-loader   CC [M]  /var/lib/dkms/falco/4.0.0+driver/build/main.o
falco-driver-loader gcc-8: error: unrecognized command line option '-mharden-sls=all'; did you mean '-mhard-float'?
falco-driver-loader make[2]: *** [scripts/Makefile.build:297: /var/lib/dkms/falco/4.0.0+driver/build/main.o] Error 1
falco-driver-loader make[1]: *** [Makefile:1906: /var/lib/dkms/falco/4.0.0+driver/build] Error 2
falco-driver-loader make[1]: Leaving directory '/host/usr/src/linux-headers-5.15.0-69-generic'
falco-driver-loader make: *** [Makefile:16: all] Error 2

[alacuku]

It seems that the falco-driver-loader does not have an up to date compiler for your kernel. Maybe @FedeDP could help us here.

[FedeDP]

Yep, unfortunately the falco-driver-loader image ships "old" gcc versions (up to gcc 8), and it is not capable of building recentish kernels like 5.0+.

falco-driver-loader The kernel was built by: gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 falco-driver-loader You are using: gcc-8 (Debian 8.3.0-6) 8.3.0

Btw kernel-crawler did find your kernel: https://falcosecurity.github.io/kernel-crawler/?arch=x86_64&target=Ubuntu&search=5.15.0-69-generic. It is also available on download.falco.org: https://download.falco.org/driver/site/index.html?lib=4.0.0%2Bdriver&target=ubuntu-generic&arch=x86_64&kind=kmod&search=falco_ubuntu-generic_5.15.0-69-generic_76.ko

You should be able to download it!

PS: the drivers for this kernel were built yesterday: https://prow.falco.org/view/s3/falco-prow-logs/logs/build-new-drivers-ubuntu-generic-5-postsubmit/1642897668080406528 :)

[poiana]

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community. /close

[poiana]

@poiana: Closing this issue.

In response to [this](https://github.com/falcosecurity/charts/issues/420#issuecomment-1534783407): >Rotten issues close after 30d of inactivity. > >Reopen the issue with `/reopen`. > >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Provide feedback via https://github.com/falcosecurity/community. >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.