search

falcosecurity / falco

Cloud Native Runtime Security
https://falco.org
Apache License 2.0
7.23k stars 893 forks source link

Plugin with field extraction capability for any event causes Falco to exit with an error in 0.32.0 #2063

Closed marc-sensenich closed 1 year ago

marc-sensenich commented 2 years ago

Describe the bug

When attempting to run Falco 0.32.0 that is configured to load plugin in development that is a field extraction plugin that supports field extraction regardless of source, Falco will exit with the following error

Error: Plugin 'ABC' has field extraction capability but is not compatible with any enabled event source

From the documentation at https://falco.org/docs/plugins/#plugin-event-sources-and-interoperability this should be allowed.

The set of compatible event sources can also be omitted. In this case, all events will be presented to the plugin, regardless of their source.

How to reproduce it

Using configuration for Falco that contains the following plugin snippet using the JSON plugin for ease of reproduction, start Falco at version 0.32.0 and encounter the following error

plugins:
  - name: json
    library_path: libjson.so
    init_config: ""

load_plugins: [json]
$ sudo falco -c ./falco.yaml 
Sat Jun 11 11:43:43 2022: Falco version 0.32.0 (driver version 39ae7d40496793cf3d3e7890c9bbdc202263836b)
Sat Jun 11 11:43:43 2022: Falco initialized with configuration file ./falco.yaml 
Sat Jun 11 11:43:43 2022: Loading plugin (json) from file /usr/share/falco/plugins/libjson.so
Error: Plugin 'json' has field extraction capability but is not compatible with any enabled event source

Expected behaviour

The expectation is that Falco should start and the field extraction plugin is then responsible for handling any event source it is given. Using the plugin stanzas from the falco.yaml snippet from "How to reproduce it" and Falco 0.31.1, Falco starts up and is ready to start handling events.

$ sudo falco -c ./falco.yaml
Sat Jun 11 11:42:28 2022: Falco version 0.31.1 (driver version b7eb0dd65226a8dc254d228c8d950d07bf3521d2)
Sat Jun 11 11:42:28 2022: Falco initialized with configuration file ./falco.yaml 
Sat Jun 11 11:42:28 2022: Loading plugin (json) from file /usr/share/falco/plugins/libjson.so
Sat Jun 11 11:42:28 2022: Loading rules from file /etc/falco/falco_rules.yaml:
Sat Jun 11 11:42:28 2022: Loading rules from file /etc/falco/falco_rules.local.yaml:
Sat Jun 11 11:42:29 2022: Starting internal webserver, listening on port 8765

Screenshots

N/A

Environment

$ falco --version
Falco version: 0.32.0
Driver version: 39ae7d40496793cf3d3e7890c9bbdc202263836b

Additional context

I believe https://github.com/falcosecurity/falco/pull/1999 introduced this potential regression, as it looks to remove the check which allowed for empty event sources for field extractor plugins.

40huo commented 2 years ago

same issue

jasondellaluce commented 2 years ago

Hi @marc-sensenich, from the example you provided it seems to be that this is actually expected behavior, and perhaps we didn't enforce proper checks on Falco v0.31.X. A way to prove my thesis is trying to load any plugin with event sourcing capability and see if Falco still reproduces this error.

The reason behind this issue is that you are loading the json plugin while running Falco with the syscall source, and as a matter of fact the json plugin is not compatible with syscall events yet. Due to to how the plugin system is currently designed, plugins with field extraction capability are not able to extract data from syscall events (and the framework blocks them from receiving those events totally). This is mostly caused by technical reasons, the first one being the fact that inspecting syscall events would also imply inspecting the libsinsp internal state, for which we don't have a clean interface yet.

See this discussion for some good explanations from @mstemm: https://kubernetes.slack.com/archives/CMWH3EH32/p1643859637793499?thread_ts=1643838967.857299&cid=CMWH3EH32 I'm also gonna link this issue for more context: https://github.com/falcosecurity/plugins/issues/85

This being said, I think allowing plugins to extract fields from syscall events is a feature we should aim for over the next months/releases, but this is not a thing yet.

marc-sensenich commented 2 years ago

Thanks for the details @jasondellaluce surrounding why we can't access the syscall source in field extraction plugins today. When loading a plugin with event sourcing enabled along with a field extraction plugin, Falco starts up as expected in 0.32.0. I'll keep a watch on the feature in future releases and will seek an alternative path to how to achieve what I was intending to do with my custom plugin. Looking for dynamic event enrichment based on Kubernetes labels in the emitted event similar to the event enrichment mentioned in the Slack conversation you've linked, https://kubernetes.slack.com/archives/CMWH3EH32/p1643838967857299.

jasondellaluce commented 2 years ago

This is starting to become quite a common use case @marc-sensenich, so we may want to bring this feature on the table soon.

juju4 commented 2 years ago

what is the config workaround for this? outside of pinning on 0.31. json plugin seems required for more than json output

my falco ansible role is not running well anymore https://github.com/juju4/ansible-falco/runs/6949597905?check_suite_focus=true#step:7:1073

jasondellaluce commented 2 years ago

Hey @juju4, I can confirm that the json plugin is not required if not for extracting fields from plugins and with plugin-dependent rulesets (such as the k8saudit one). Why are you loading the json plugin in this case? Can you share your falcio.yaml config or give a pointer to it?

Either way, the workaround is just to not load the plugin if you don't need it. It would be load_plugins: [] in falco.yaml.

juju4 commented 2 years ago

One example here https://github.com/juju4/ansible-falco/runs/7053893404?check_suite_focus=true#step:7:1081 config https://github.com/juju4/ansible-falco/runs/7053893404?check_suite_focus=true#step:11:78

my initial reason was to get json output but even if it is disabled, still getting the error.

slapula commented 2 years ago

I'm also running into this issue (using @juju4 ansible role) whoever I'm blocked on falco not having 0.31 built for arm64 at the moment. Not a huge deal as this a home lab I'm working on but I do want to chime in, confirm this issue, and volunteer if anyone needs testing/validation etc.

juju4 commented 2 years ago

any implementable workaround? for now, pinning on 0.31 and role github actions failing every week :(

jasondellaluce commented 2 years ago

@juju4 Looking at the workflow output. Seems like you are trying to validate the config in falco.yaml.j2, and then putting it inside /etc/falco/falco.yaml. However, what Falco fails to validate is falco.yaml.j2, and since that that check is failed I don't think it gets actually copied in /etc/falco/falco.yaml. In fact, the configuration printout you linked is basically the currently default one, which is not what's inside falco.yaml.j2.

I see falco.yaml.j2 was lastly updated ~2 years ago. I think it's worth trying to update it according to the latest falco.yaml of Falco, by making sure both the plugins : ... and load_plugins: [] entries are include. Let me know how that goes.

juju4 commented 2 years ago

devel branch falco.yaml.j2 is from 2 months ago. https://github.com/juju4/ansible-falco/tree/devel/templates

I'm using ansible validate option which is assuming than falco is not requiring reference config file to be in a single place. This avoids to install a faulty config. The end print config is only relevant if successful else it is the default one.

I'm not currently interested in json sources but I want json log output (json_output but even if false, there is error, like below). not sure if that needs json plugin or not.

here a branch results with empty plugins and no config validation and problematic config file https://github.com/juju4/ansible-falco/runs/7819504087?check_suite_focus=true#step:7:19368 https://github.com/juju4/ansible-falco/runs/7819504087?check_suite_focus=true#step:11:60

jasondellaluce commented 2 years ago

Now you're getting Error: Plugin 'json' is required but not loaded, which is a different error from before. This error is due to the /etc/falco/k8s_audit_rules.yaml being loaded, which requires the k8s_audit and json plugins and has no use without them.

Remove /etc/falco/k8s_audit_rules.yaml from your falco.yaml config and I think it should load with no errors.

juju4 commented 2 years ago

ok, so without any plugins and k8 rules, pipeline works again. Thanks https://github.com/juju4/ansible-falco/actions/runs/2852352663

if I try a config with plugins json and k8s_audit and k8s rules, back to same error. https://github.com/juju4/ansible-falco/runs/7819993610?check_suite_focus=true#step:7:1081 Does it need extra config? Or can this be used only on real k8 cluster?

poiana commented 1 year ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

poiana commented 1 year ago

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

poiana commented 1 year ago

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community. /close

poiana commented 1 year ago

@poiana: Closing this issue.

In response to [this](https://github.com/falcosecurity/falco/issues/2063#issuecomment-1423283863): >Rotten issues close after 30d of inactivity. > >Reopen the issue with `/reopen`. > >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Provide feedback via https://github.com/falcosecurity/community. >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.