search

falcosecurity / falco

Cloud Native Runtime Security
https://falco.org
Apache License 2.0
7.14k stars 880 forks source link

Falco Unable to insmod module on Amazon Linux 2 EKS #3102

Open rama-akbar opened 5 months ago

rama-akbar commented 5 months ago

Describe the bug

We want to install Falco into our EKS cluster, but got an Unable to insmod module error.

Expected behaviour

Falco running sucess

Error Message

2024-02-19 12:26:52 INFO  Running falcoctl driver install
                      ├ driver version: 7.0.0+driver
                      ├ driver type: kmod
                      ├ driver name: falco
                      ├ compile: true
                      ├ download: true
                      ├ arch: x86_64
                      ├ kernel release: 5.10.198-187.748.amzn2.x86_64
                      └ kernel version: #1 SMP Tue Oct 24 19:49:54 UTC 2023
2024-02-19 12:26:52 INFO  Found distro target: amazonlinux2
2024-02-19 12:26:52 INFO  Check if kernel module is still loaded.
2024-02-19 12:26:52 INFO  OK! There is no module loaded.
2024-02-19 12:26:52 INFO  Check all versions of kernel module in dkms.
2024-02-19 12:26:52 INFO  OK! There are no module versions in dkms.
2024-02-19 12:26:52 INFO  Trying to download a driver.
                      └ url: https://download.falco.org/driver/7.0.0%2Bdriver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko
2024-02-19 12:26:54 INFO  Driver downloaded.
                      └ path: /root/.falco/7.0.0+driver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko
2024-02-19 12:26:54 WARN  Unable to insmod module.
                      ├ driver: /root/.falco/7.0.0+driver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko
                      └ err: exit status 1

Environment

FedeDP commented 5 months ago

Hi! Thanks for opening this issue! Can you manually try insmod /root/.falco/7.0.0+driver/x86_64/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko and paste here the output?

FedeDP commented 5 months ago

/assign

rama-akbar commented 5 months ago

@FedeDP

Output of insmod, we deploy it using helm chart from official falco - https://github.com/falcosecurity/charts/blob/master/charts/falco/values.yaml

insmod: ERROR: could not insert module root/falco_amazonlinux2_5.10.198-187.748.amzn2.x86_64_1.ko: Operation not permitted
FedeDP commented 5 months ago

Mmh it seems like you are missing some permissions; how did you deploy the Falco helm chart? Did you modify anything from default values?

rama-akbar commented 5 months ago

Hi @FedeDP here is diff

left = ours right = current chart https://github.com/falcosecurity/charts/blob/master/charts/falco/values.yaml

https://www.diffchecker.com/HPWzTzsU/

Is there posibility latest version of falco didn't support on Amazon Linux 2 ?

FedeDP commented 5 months ago

Mmmh from your diff, it seems like you are using ebpf driver?

kind: ebpf ebpf: path:

But why did you clear up the path? Also, your OP showed that falcoctl tried to install kmod instead (and indeed the error is about insmod).

Is there posibility latest version of falco didn't support on Amazon Linux 2 ?

Nope, we support amazonlinux2.

rama-akbar commented 5 months ago

Mmmh from your diff, it seems like you are using ebpf driver?

kind: ebpf ebpf: path:

But why did you clear up the path? Also, your OP showed that falcoctl tried to install kmod instead (and indeed the error is about insmod).

I'm not sure actually, because chart we used is chart when we deploy falco version 0.34.1, if we deploy falco 0.34.1 it works fine.

but when we deploy latest version 0.37.1, we getting the kernel module issue

poiana commented 2 months ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana commented 1 month ago

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

hitsub2 commented 2 weeks ago

Same issue here with Bottlerocket

EKS 1.29 OS: Bottlerocket

helm install command

helm install falco falcosecurity/falco
--create-namespace \
--namespace falco \
--set tty=true \
--set collectors.kubernetes.enabled=true \ 
--set driver.kind=kmod

error logs by kubectl logs falco-4dtwz -n falco -c falco-driver-loader

* Setting up /usr/src links from host
2024-07-11 02:02:06 INFO  Running falcoctl driver config
                      ├ name: falco
                      ├ version: 7.2.0+driver
                      ├ type: kmod
                      ├ host-root: /host
                      └ repos: https://download.falco.org/driver
2024-07-11 02:02:06 INFO  Storing falcoctl driver config
2024-07-11 02:02:06 INFO  Running falcoctl driver install
                      ├ driver version: 7.2.0+driver
                      ├ driver type: kmod
                      ├ driver name: falco
                      ├ compile: true
                      ├ download: true
                      ├ target: bottlerocket
                      ├ arch: x86_64
                      ├ kernel release: 6.1.92
                      └ kernel version: #1 SMP PREEMPT_DYNAMIC Mon Jun 24 17:51:17 UTC 2024
2024-07-11 02:02:06 INFO  Check if kernel module is still loaded.
2024-07-11 02:02:06 INFO  OK! There is no module loaded.
2024-07-11 02:02:06 INFO  Check all versions of kernel module in dkms.
2024-07-11 02:02:06 INFO  OK! There are no module versions in dkms.
2024-07-11 02:02:06 INFO  Trying to download a driver.
                      └ url: https://download.falco.org/driver/7.2.0%2Bdriver/x86_64/falco_bottlerocket_6.1.92_1_1.20.3-aws.ko
2024-07-11 02:02:08 INFO  Driver downloaded.
                      └ path: /root/.falco/7.2.0+driver/x86_64/falco_bottlerocket_6.1.92_1_1.20.3-aws.ko
2024-07-11 02:02:08 WARN  Unable to insmod module.
                      ├ driver: /root/.falco/7.2.0+driver/x86_64/falco_bottlerocket_6.1.92_1_1.20.3-aws.ko
                      └ err: exit status 1
FedeDP commented 1 week ago

Hi! Since this is a recent kernel release, the preferred method is to use the modern_ebpf driver that does not need any external artifact. In the meantime i'll investigate about why we are failing to inject kmod there; thanks for reporting!