search

falcosecurity / plugin-sdk-go

Falco plugins SDK for Go
Apache License 2.0
26 stars 17 forks source link

Increase the default event limit to 256k #22

Closed mstemm closed 3 years ago

mstemm commented 3 years ago

Plugin events are no longer capped at 64k, and we think 64k might be too small for some plugins, so change the default to 256k.

Signed-off-by: Mark Stemm mark.stemm@gmail.com

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area plugin-sdk

/area tests

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE
jasondellaluce commented 3 years ago

My only concern is the memory footprint of the event batches. The default reusable event batch will allocate a memory buffer of size DefaultBatchSize * DefaultEvtSize, which until now was about ~32MBs. With this change, every source plugin would use ~128MBs of memory by default, which I think is a bit excessive.

Should we also decrease DefaultBatchSize to 64 or 128? Other than that, this looks fine to me!

leogr commented 3 years ago

My only concern is the memory footprint of the event batches. The default reusable event batch will allocate a memory buffer of size DefaultBatchSize * DefaultEvtSize, which until now was about ~32MBs. With this change, every source plugin would use ~132MBs of memory by default, which I think is a bit excessive.

Should we also decrease DefaultBatchSize to 64 or 128? Other than that, this looks fine to me!

I agree. My vote goes to decrease DefaultBatchSize to 128 (so we will get the same memory footprint, ie. 128 x 256k = ~32MB). Although this choice is a bit conservative, we can reserve to perform an accurate benchmark and then fine-tune the defaults again.

Moreover, IMHO it is worth highlighting more in the documentation that these parameters can be easily configured by the plugin author.

mstemm commented 3 years ago

Okay, I decreased the number of events and kept the event size at 256k.

poiana commented 3 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: leogr, mstemm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/falcosecurity/plugin-sdk-go/blob/main/OWNERS)~~ [leogr,mstemm] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
poiana commented 3 years ago

LGTM label has been added.

Git tree hash: 314c5528eb6c1febfdbe9ed1268660019a4a9ff4