Added logging the best Netty's maxOrder setting when big payload trigger an unpooled allocation
What does this PR do?
Print a log line with the preferred Netty's maxOrder to accomodate the buffer size that exceeds the actual chunk size that would trigger an unpooled allocation.
It prints the log just one time across all plugin instances for each maxOrder required. So if plugin instance A reaches the limit of 4Mb (aka maxOrder 9) with a desired payload size of 7Mb it will report line about to use maxOrder 10. After this log line is printed, any other plugin instance wouldn't report it again, unless it needs an even bigger maxOrder not yet reported.
This PR also introduces a test spy of the Logger class to test the behaviour.
Why is it important/What is the impact to the user?
Helps the user to set the right maxOrder for his message flow to avoid slow unpooled allocation.
Checklist
[x] My code follows the style guidelines of this project
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have made corresponding change to the default configuration files (and/or docker env variables)
[x] I have added tests that prove my fix is effective or that my feature works
Author's Checklist
[x] add test that when pass maxOrder n doesn't print either notification for < n
run the benchmark tool to create payloads > 4MB, from ingest_all_hands_ams run:
bundle exec ruby -J-Xmx16g -J-XX:-MaxFDLimit benchmark_client.rb --test=beats --msg_sizes=2500 -a yes
verify in the logs of Logstash, it report the preferred maxOrder of 10, like:
[2024-02-14T14:18:07,863][WARN ][org.logstash.beats.V2Batch][beats_input][32f207bacda794ed2bf54ea29a1645a3dbb680398856e5cabef16bde58104d8d][defaultEventExecutorGroup-4-5]
Got a batch size of 4194342 bytes, while this instance expects batches up to 4194304, please bump maxOrder to 10.
Release notes
Added logging the best Netty's
maxOrder
setting when big payload trigger an unpooled allocationWhat does this PR do?
Print a log line with the preferred Netty's
maxOrder
to accomodate the buffer size that exceeds the actual chunk size that would trigger an unpooled allocation. It prints the log just one time across all plugin instances for each maxOrder required. So if plugin instance A reaches the limit of 4Mb (aka maxOrder 9) with a desired payload size of 7Mb it will report line about to use maxOrder 10. After this log line is printed, any other plugin instance wouldn't report it again, unless it needs an even bigger maxOrder not yet reported. This PR also introduces a test spy of the Logger class to test the behaviour.Why is it important/What is the impact to the user?
Helps the user to set the right
maxOrder
for his message flow to avoid slow unpooled allocation.Checklist
[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files (and/or docker env variables)Author's Checklist
n
doesn't print either notification for< n
How to test this PR locally
It needs the test tool present in branch
ingest_all_hands_ams
contained in PR https://github.com/elastic/logstash/pull/15151../gradlew vendor
)config/(log4j2.properties
) add:ingest_all_hands_ams
run:maxOrder
of 10, like:Related issues
Use cases
Screenshots
Logs