search

falcosecurity / falcosidekick

Connect Falco to your ecosystem
Apache License 2.0
547 stars 178 forks source link

Missing customfields/extra_fields on Elasticsearch export #1033

Open qsoul opened 6 days ago

qsoul commented 6 days ago

Describe the bug

When using falco's append_output.extra_fields or falcosidekick customfields they both appear in falco's output, but are missing when exporting to Elasticsearch (via Elasticsearch output method). Those extra fields just are not present at all in the post request json body.

Issif commented 5 days ago

Hi,

I think I was able to replicate the issue. Can you confirm me that the value elasticsearch.flattenfields is true in your config?

Edit: there's a bug in my code to "flatten" the keys of the output_fields, a check is missing, and the keys without any dot are removed. I'm on the fix right now

Issif commented 5 days ago

The PR with the fix is https://github.com/falcosecurity/falcosidekick/pull/1034/files, it will be in the next 2.30, sadly I don't have any ETA, but I could generate a release candidate for you if you need.

qsoul commented 4 days ago

Hi,

I think I was able to replicate the issue. Can you confirm me that the value elasticsearch.flattenfields is true in your config?

Edit: there's a bug in my code to "flatten" the keys of the output_fields, a check is missing, and the keys without any dot are removed. I'm on the fix right now

Thank you for handling it so fast! I'll use the lastest image untill the next release. One more thanks! p.s. I don't use elasticsearch.flattenfields explicitly, but use createindextemplate: true, so yes, that should be same I believe.

Issif commented 4 days ago

Yes, the createtemplate setting implies it, to avoid conflicts with some fields types that may appear as nested or not, we got multiple issues because of that