Event dependent dictionary_path

[svetli-n]

Hi and thanks for the nice plugin!

How about having dependent dictionary_path, for example dictionary_path => "%{host}.yml"

Something similar: https://github.com/logstash-plugins/logstash-output-file/blob/master/lib/logstash/outputs/file.rb#L83

[snowdevil17]

+1, I have an environment where I need the path to change dependent on time. The data is not real time, so i need to have a history of dictionaries for bulk historic imports.

edit: to be clear, I don't need it to match ON time, i use the range logstash filter for that. I translate a timeframe into a specific identifier in a field. Then hoped to use %{field}.json for using the right dictionary.

[sethyes]

Similarly, I have a case where different translations need to be ran depending the source of the event. Therefore, being able to have a dictionary_path => "event_translate_%{source}" would be extremely useful, rather than having to use case statements like a caveman.

[jordansissel]

I think this idea sounds reasonable.

On Mon, Oct 30, 2017 at 3:09 PM Seth Smith notifications@github.com wrote:

Similarly, I have a case where different translations need to be ran depending the source of the event. Therefore, being able to have a dictionary_path => "eventtranslate%{source}" would be extremely useful, rather than having to use case statements like a caveman.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/logstash-plugins/logstash-filter-translate/issues/18#issuecomment-340599925, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIC6oBCmQcZsiko-nBTbP7C-fyWP_HFks5sxkkfgaJpZM4HYJpX .

[wols]

It would also make my job a lot easier, using the same key several times.

# dict_00
A.B.C.D: alias_zero_zero
...
# dict_99.yml
A.B.C.D: alias_nine_nine
...

if "..." { mutate { replace => { "dict_file" => "dict_01" } } }
else if "..." {  } # and so on
else if "..." { mutate { replace => { "dict_file" => "dict_99" } } }
else "..." { mutate { replace => { "dict_file" => "dict_00" } } }

translate {
    ...
    dictionary_path => "/etc/logstash/dictionary/%{dict_file}.yml"
    ...
}

[guyboertje]

This idea is not feasible given that the file is loaded when LS starts. I'm also about to add a pull request that changes the dictionary refresh to being async from the filter invocation cadence.

My advice is to use the jdbc_static filter. The parameters setting allows you to build a interpolated key, e.g. parameters => { "sought" => "%{[host]}-%{[some_id]}" } To achieve this you need to have all the different domain rows in one table with a domain + id column to match by.