My use case is fairly typical: call age{} in the logstash filter and then compare the target field against some threshold. I could not stand the hard coded magic number limit used in the comparison so I forked the repo and developed a POC enhancment that may result in a subsequent pull request.
I added these config fields:
"@url" to limit service returning a json response containing a limit field.
I use elasticsearch cluster settings metadata with great success.
"@user" "@password" basic auth credentials for the limit service
"@expired_target" name of a boolean target field to store the results of the age diff against limit field comparison
"@frequency" how many seconds between scheduled calls the limit service to get the latest limit field (in case it changed).
@age_limit is an instasnce member to cache the latest limit value.
I used the http_client mixin and I'm folding in the rufus_scheduler gem to do the hard work.
The age class register method makes the http request to the limit service synchronously and starts a periodic schedule to perform the next get request for the age limit.
I am new to logstash and filters and ruby, but so far this is working great.
I have concerns about the statefulness of the class instance member access in a multi worker logstash environment. Out of an abundance of caution I am considering adding require require 'jruby/synchronized' and then including JRuby::Synchronized as per https://github.com/jruby/jruby/wiki/Concurrency-in-jruby
I wonder if this enhancement would be useful as a PR or if I should leave it as a fork or develop a completely new alternative to age.
richard-mauri
commented
4 years ago
My use case is fairly typical: call age{} in the logstash filter and then compare the target field against some threshold. I could not stand the hard coded magic number limit used in the comparison so I forked the repo and developed a POC enhancment that may result in a subsequent pull request.
I added these config fields:
I used the http_client mixin and I'm folding in the rufus_scheduler gem to do the hard work.
The age class register method makes the http request to the limit service synchronously and starts a periodic schedule to perform the next get request for the age limit.
I am new to logstash and filters and ruby, but so far this is working great. I have concerns about the statefulness of the class instance member access in a multi worker logstash environment. Out of an abundance of caution I am considering adding require require 'jruby/synchronized' and then including JRuby::Synchronized as per https://github.com/jruby/jruby/wiki/Concurrency-in-jruby
I wonder if this enhancement would be useful as a PR or if I should leave it as a fork or develop a completely new alternative to age.
Thoughts?