logstash-plugins / logstash-filter-translate

Translate filter for Logstash
Apache License 2.0
21 stars 47 forks source link

Add JDBC Loader code from jdbc_static filter to load a dictionary #68

Open guyboertje opened 6 years ago

guyboertje commented 6 years ago

It should be fairly simple to extract the loader code from the jdbc_static filter and add it as a means of building a dictionary.

Some tweaks would be required to specific which column is the dictionary key and whether to support a "rich" object as the value or a concatenated string that dissect could deconstruct into the Event root.

This idea is inspired by the fact that jdbc_static is very slow for huge datasets and it is more suited for complex SQL work rather than simple key lookup.