Falco installation an AWS EKS with Amazon Linux 2 image

[ajinkya1986]

Describe the bug We are trying to install Falco in Amazon EKS with Amazon Linux 2 image, but the ebpf probe is not getting installed and cannot be downloaded

Trying to download a prebuilt eBPF probe from https://download.falco.org/driver/39ae7d40496793cf3d3e7890c9bbdc202263836b/fal ││ curl: (22) The requested URL returned error: 404  
Unable to find a prebuilt falco eBPF probe                                                                                     
* Trying to compile the eBPF probe (falco_amazonlinux2_5.4.238-148.346.amzn2.x86_64_1.o)                                       
*  expr: syntax error: unexpected argument '1'                                                                                    
* make[1]: *** /lib/modules/5.4.238-148.346.amzn2.x86_64/build: No such file or directory.  Stop.                                
* make: *** [Makefile:38: all] Error 2                                                                                           
* mv: cannot stat '/usr/src/falco-39ae7d40496793cf3d3e7890c9bbdc202263836b/bpf/probe.o': No such file or directory               │Unable to load the falco eBPF probe 

How to reproduce it Create a standard EKS cluster and install falco.

Environment

• Falco version: 0.32
• System info:
• Cloud provider or hardware configuration:
• AWS
• OS: AL2_x86_64
• Kernel: 5.4.238-148.346
• Installation method: helm

[Andreagit97]

it seems like you miss kernel headers on your node :/ you can try something like sudo yum install -y kernel-devel-$(uname -r)

[ajinkya1986]

@Andreagit97 we do not have access to the node shell

[Andreagit97]

uhm ok, so the only way you have is to rely on a pre-built driver, it seems we already have an issue for that on test-infra repository https://github.com/falcosecurity/test-infra/issues/1093

[mpurusottamc]

@Andreagit97 thanks for reference. Facing similar issue for falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1 as well.

I can see that there's a pre-built driver here - https://download.falco.org/driver/site/index.html?lib=4.0.0%2Bdriver&target=amazonlinux2&arch=x86_64&kind=kmod&search=falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.ko

But, I still see an error. In our falco setup, it's looking for a .o file. but, i don't see the .o file. I only see .ko file. Any suggestions how to address this?

[mpurusottamc]

@Andreagit97 :(. My filter was incorrect. I can see the .o file when selected the ebpf filter.

I can see the file here - (https://download.falco.org/driver/site/index.html?lib=4.0.0%2Bdriver&target=amazonlinux2&arch=x86_64&kind=ebpf&search=falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o)

But, still i see the error. Any suggestions?

[Andreagit97]

@mpurusottamc Could you share the Falco output?

[mpurusottamc]

@Andreagit97 Here is the log:

* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.32.0, driver version=39ae7d40496793cf3d3e7890c9bbdc202263836b
* Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
* Mounting debugfs
* Trying to download a prebuilt eBPF probe from https://download.falco.org/driver/39ae7d40496793cf3d3e7890c9bbdc202263836b/falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o
curl: (22) The requested URL returned error: 404
Unable to find a prebuilt falco eBPF probe
* Trying to compile the eBPF probe (falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o)
expr: syntax error: unexpected argument '1'
make[1]: *** /lib/modules/5.10.173-154.642.amzn2.x86_64/build: No such file or directory.Stop.
make: *** [Makefile:38: all] Error 2
mv: cannot stat '/usr/src/falco-39ae7d40496793cf3d3e7890c9bbdc202263836b/bpf/probe.o': No such file or directory
Unable to load the falco eBPF probe
Wed Apr 12 20:12:22 2023: Falco version 0.32.0 (driver version 39ae7d40496793cf3d3e7890c9bbdc202263836b)
Wed Apr 12 20:12:22 2023: Falco initialized with configuration file /etc/falco/falco.yaml
Wed Apr 12 20:12:22 2023: Loading rules from file /etc/falco/falco_rules.yaml:
Wed Apr 12 20:12:23 2023: Loading rules from file /etc/falco/falco_rules.local.yaml:
Wed Apr 12 20:12:23 2023: Starting internal webserver, listening on port 8765
Wed Apr 12 20:12:23 2023: Unable to load the driver.
Wed Apr 12 20:12:23 2023: Runtime error: can't open BPF probe '/root/.falco/falco-bpf.o': Errno 2. Exiting.

I see the mistake I did. The falco version that is installed is 0.32.0 with driver version 39ae7d40496793cf3d3e7890c9bbdc202263836b. But, I was searching for drivers for 4.0.0+driver version.

ebpf probe for this os (falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64) for falco driver version 39ae7d40496793cf3d3e7890c9bbdc202263836b does not exist. But, ebpf probe for this os (falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64) for falco driver version 4.0.0+driver exists

I am assuming, if an ebpf probe exists for a different driver version, we should update the driver version to make it work. Is this a fair understanding?

Now, I am curious 😄 , If there's no ebpf probe for a specific linux version across all driver versions, how should that be resolved?

[FedeDP]

Yep! You just need to update Falco to latest version :) We only officially support latest 3 driver versions. Thus we only provide updates for prebuilt drivers for them!

[mpurusottamc]

that explains it. Thanks for your help @FedeDP @Andreagit97

[ajinkya1986]

Thanks for the response @FedeDP and @Andreagit97

[ajinkya1986]

Hi just wanted to update. We have tested the falco versions 2.0.18, 2.5.5, 3.0.0 and 3.1.3 with EKS kubernetes versions 1.22, 1.23,1.24,1.25 and 1.26. The falco daemonset is working now as expected with successful loading of the ebpf probe and we are getting the threats data as well. @mpurusottamc @FedeDP @Andreagit97

[FedeDP]

That's great! This is surely linked to https://github.com/falcosecurity/test-infra/issues/1093; please leave a comment there if that's the case :)

[ajinkya1986]

Ok Sure @FedeDP

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[poiana]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

[Andreagit97]

can we close this?

[ajinkya1986]

@Andreagit97 Yes you can close it. Thank you for your support.