Unable to load eBPF probe for Amazon Linux 5.10.173-154.642

[mpurusottamc]

Describe the bug I am trying to install falco helm chart on an Amazon Linux machine and getting the below error.

* Setting up /usr/src links from host
* Running falco-driver-loader for: falco version=0.32.0, driver version=39ae7d40496793cf3d3e7890c9bbdc202263836b
* Running falco-driver-loader with: driver=bpf, compile=yes, download=yes
* Mounting debugfs
* Trying to download a prebuilt eBPF probe from https://download.falco.org/driver/39ae7d40496793cf3d3e7890c9bbdc202263836b/falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o
curl: (22) The requested URL returned error: 404
Unable to find a prebuilt falco eBPF probe
* Trying to compile the eBPF probe (falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o)
expr: syntax error: unexpected argument '1'
make[1]: *** /lib/modules/5.10.173-154.642.amzn2.x86_64/build: No such file or directory.Stop.
make: *** [Makefile:38: all] Error 2
mv: cannot stat '/usr/src/falco-39ae7d40496793cf3d3e7890c9bbdc202263836b/bpf/probe.o': No such file or directory
Unable to load the falco eBPF probe
Wed Apr 12 20:12:22 2023: Falco version 0.32.0 (driver version 39ae7d40496793cf3d3e7890c9bbdc202263836b)
Wed Apr 12 20:12:22 2023: Falco initialized with configuration file /etc/falco/falco.yaml
Wed Apr 12 20:12:22 2023: Loading rules from file /etc/falco/falco_rules.yaml:
Wed Apr 12 20:12:23 2023: Loading rules from file /etc/falco/falco_rules.local.yaml:
Wed Apr 12 20:12:23 2023: Starting internal webserver, listening on port 8765
Wed Apr 12 20:12:23 2023: Unable to load the driver.
Wed Apr 12 20:12:23 2023: Runtime error: can't open BPF probe '/root/.falco/falco-bpf.o': Errno 2. Exiting.

How to reproduce it

Create a new EKS cluster with below scripts.

1. create eks-cluster.yml file with below config

   
   apiVersion: eksctl.io/v1alpha5
   kind: ClusterConfig

metadata: name: eks-local-testing-cluster region: us-east-2

nodeGroups:

• name: ng-1 instanceType: t3.large desiredCapacity: 1 volumeSize: 50

• name: ng-2 instanceType: t3.large desiredCapacity: 1 volumeSize: 50

2. create the cluster with below command:

   eksctl create cluster -f eks-cluster.yaml

Expected behaviour Was expecting the falco service to start without errors.

Environment Falco version: 0.32.0 System info: Cloud provider or hardware configuration: AWS OS: AL2_x86_64 Kernel: falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64 Installation method: helm Helm chart version: 1.19.1

Additional Information I can see the ebpf probe file here - (https://download.falco.org/driver/site/index.html?lib=4.0.0%2Bdriver&target=amazonlinux2&arch=x86_64&kind=ebpf&search=falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o)

Any suggestions would be much appreciated. Thanks a lot.

[tspearconquest]

Not sure why the downloader is trying to get it from a different path but you could try to download it manually, put it into a new docker layer which contains the ebpf probe file at the path /root/.falco/4.0.0+driver/x86_64/falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o and add that layer on top of the existing falco driver loader image.

[tspearconquest]

The above is not recommended for prod usage; just troubleshooting purposes

[mpurusottamc]

@tspearconquest The downloader is trying to get the right version https://download.falco.org/driver/39ae7d40496793cf3d3e7890c9bbdc202263836b/falco_amazonlinux2_5.10.173-154.642.amzn2.x86_64_1.o

But, that does not exist.

the 4.0.0+driver version was more of a manual search that i did in the probes listing page.

If a probe does not exist for a particular version, how should that be resolved in production?

[tspearconquest]

The falco team would need to release it or you would need to compile it yourself. It appears that falco did try to compile it but encountered expr: syntax error: unexpected argument '1' so I will have to defer to them to troubleshoot this error, as I'm not in AWS and couldn't help to make a release (I'm a user here too :) ).

[mpurusottamc]

@Andreagit97 any suggestions?

[FedeDP]

As I wrote in #2488 , we only support updates to prebuilt drivers for latest 3 driver versions. Therefore you need to update your Falco installation to latest one (ie: the one that uses driver 4.0.0) to be able to fetch the prebuilt driver.

[mpurusottamc]

Thanks for your help @FedeDP. We will update and test it.