Provide ability in config file to declare a variable that can be referenced throughout the file (e.g. pattern_dir)

[woolfie]

In a large logstash config file, I often find myself having to specify the same pattern_dir for the grok filter multiple times throughout the file.

It would be very helpful if I could set this path once at the top of the config file and then refer to it throughout the file.

This is just one example where the ability to set and reference a variable would be useful, I'm sure there are others.

[jordansissel]

Given the wide variety of config management options, I'm not sure we should implement this.

Your current options for this are:

• use a templating system (m4, for example) to achieve what you need.
• or, use a config management tool (puppet, chef, etc) to help you automate this.

I do not believe logstash itself needs to support what is already provided quite well by other complimentary tools, but am willing to hear arguments to the contrary.

I strongly recommend you implement config management (like puppet or chef) for your whole infrastructure - but such a recommendation is obviously outside the scope of this ticket :)

[woolfie]

I'll have to look into the tools you recommend, as right now we don't use any of those, however pulling in a whole other tool layer just so I can use variables seems... overkill?

But yes, I will still look into them. Thanks.

[realpdm]

I would like to see a way to set the pattern dir once. Even with a templating system it seems kind of silly to set the pattern dir once for every filter. I don't like the idea of variable setting, but I do like the idea of a global setting for some filters where it makes sense. Having an global user config pattern directory seems like an useful optimization to make.

Maybe this suggestion needs to be its own issue.

[electrical]

Hi,

While it might make sense in some cases, it would mean that we put a plugin specific setting into the core application. Since the Core and plugins need to be able to function completely separately of each other having that crossing of settings is something we don't want.

With puppet for example one can use a template for the grok filter which has the patterns dir hard coded or as a variable and define the variable some where else.

[suyograo]

Closed based on @jordansissel comments

[grahamoregan]

This is something that seems to keep coming up (from googling for a solution), one case where a templated file won't work is deploying logstash in an asg on ec2 with prebaked AMIs - the endpoints may not be known at the time that the AMI is created but UserData could populate the environment.

[reinaldovale]

Não sei se é thread safe. @metadata somente no logstash 1.5. Retire isso em versões anteriores.

ruby{ code => "$data_atualizacao = $data_atualizacao ? $data_atualizacao : (event['[@metadata][data_atualizacao]'].to_s) event['data_atualizacao'] = $data_atualizacao" }

[blindpet]

@jordansissel do you have any resources for using ansible, puppet or check config template variables with logstash?

We have some SAN switches we need to run an exec command on from logstash and process the output, we are losing the IP of the SAN node in the process.

[bozernov]

Being able to set a variable will give possibility to use it in multiline - to join lines which have the same timestamps for example.