magnusbaeck / logstash-filter-verifier

Apache License 2.0
192 stars 27 forks source link

Environment variables not passed through to logstash #86

Closed jgough closed 4 years ago

jgough commented 4 years ago

It looks like environment variables aren't being passed through to logstash when it is run. For example with this config file:

input { generator { lines => ["msg"] count => 1 } }
filter { mutate { add_field=>{"myvar"=>"${MYVAR}" } } }
output { stdout { codec => rubydebug } }

When I run MYVAR=foo logstash -f ./test.conf

I get the correct output:

{
         "myvar" => "foo",
       "message" => "msg",
      "@version" => "1",
    "@timestamp" => 2020-07-23T16:25:36.113Z,
          "host" => "a8bef8d96fb6",
      "sequence" => 0
}

If I run MYVAR=foo logstash-filter-verifier test.json test.conf

Then logstash exits with an error:

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Cannot evaluate `${MYVAR}`. Replacement variable `MYVAR` is not defined in a Logstash secret store or as an Environment entry and there is no default value given."

I'm actually trying to use variables to change the config values in a test environment so it would be good if this were supported.

magnusbaeck commented 4 years ago

The environment variables are scrubbed so they won't affect the test execution. To explicitly indicate that an environment variable should be passed on to the Logstash child invoke LFV with --keep-env MYVAR. I should document this in the readme file so people won't have to read the --help fine print.

jgough commented 4 years ago

Sorry, didn't spot that option in there! Thanks for all your work on this :-)