logstash-plugins / logstash-filter-jdbc_static

Loads a DB query's result set in memory and uses it as lookup table for events.
Apache License 2.0
11 stars 12 forks source link

Loader specific refresh policy #14

Open acchen97 opened 6 years ago

acchen97 commented 6 years ago

The loader_schedule option today is global across all the loaders specified in a plugin instance. This prohibits a user from setting different cache refresh policies for different loaders (and effectively local tables). Implementing loader_schedule option for each loader separately would enable this finer grained tuning.

filter {
  jdbc_static {
    loaders => [ 
      {
        id => "remote-servers"
        query => "select ip, descr from ref.local_ips order by ip"
        local_table => "servers"
        loader_schedule => "0 22 * * *"
      },
      {
        id => "remote-users"
        query => "select firstname, lastname, userid from ref.local_users order by userid"
        local_table => "users"
      }
    ]
    ...
jakelandis commented 6 years ago

I may be easier to implement multiple schedulers via Java. I have used http://www.quartz-scheduler.org/ in prior projects with good success. (it also knows how to parse a cron string)