bernd
commented
9 years ago It is hard to say what the limiting factor on your machines is. That totally depends on the message load (number msgs/s, message size), configured extractors, configured streams, Java heap settings, CPU cores, graylog2.conf settings and ES performance.
You can tweak the settings for disabling the streams with the following two settings.
# Every message is matched against the configured streams and it can happen that a stream contains rules which
# take an unusual amount of time to run, for example if its using regular expressions that perform excessive backtracking.
# This will impact the processing of the entire server. To keep such misbehaving stream rules from impacting other
# streams, Graylog limits the execution time for each stream.
# The default values are noted below, the timeout is in milliseconds.
# If the stream matching for one stream took longer than the timeout value, and this happened more than "max_faults" times
# that stream is disabled and a notification is shown in the web interface.
stream_processing_timeout = 2000
stream_processing_max_faults = 3 You can either increase the timeout or the max faults settings. (setting the max faults to 0 will disable the stream pausing completely)
I hope that helps. Please re-open the issue if you still have problems after playing with the settings. Thank you!
Hello
I have had numerous problems with my installation of 1.0.0 stable having its streams auto paused all the time. I fired up a brand new m3.medium EC2 instance hosting only Graylog server (blank install) and web interface, my streams are still always being paused:
When I login to the control panel, I see this:
Also getting this:
I have a few streams that all check the level (for errors) and then have a regex to check a few hostnames, ie:
What is lacking in the default setup of Graylog that is causing it to behave and perform so badly? Can anyone provide any tips?
Thankyou.