Currently logstash-filter-verifier starts a new Logstash instance for every test case file. This is mainly due to the fact, that logstash-input-stdin is used as the input. If the test cases use different configuration for the input (codec, fields), there is no other way with the logstash-input-stdin plugin other than restart Logstash for every test case file.
One of the main advantages of the logstash-input-stdin plugin is, that it can be shut down from the outside (by the controlling process) by closing stdin, if it is the only input plugin running within Logstash.
After all input plugins are properly shut down, Logstash terminates after successful processing of all events. This allows the controlling process to properly determine, if Logstash has finished its work.
A short analysis of the Logstash input plugins showed, that there are not a lot of other plugins in the Logstash eco system, which share this behavior. One of the remaining possibilities would be logstash-input-unix plugin, if it is run in mode => "client". In this case, the sending party (controlling process) is also able to shut down the logstash-input-unix plugin. The logstash-input-unix plugin may be used several times for a single Logstash instance, which allows to use one Logstash instance for all tests, which greatly improves the total duration for the tests, as Logstash is very slow to start compared to the effective time used for the tests.
I have put together some lines, which extend logstash-filter-verifier to allow the usage of the logstash-input-unix plugin as input and therefore allows to run the tests in parallel.
Currently logstash-filter-verifier starts a new Logstash instance for every test case file. This is mainly due to the fact, that logstash-input-stdin is used as the input. If the test cases use different configuration for the input (codec, fields), there is no other way with the logstash-input-stdin plugin other than restart Logstash for every test case file.
One of the main advantages of the logstash-input-stdin plugin is, that it can be shut down from the outside (by the controlling process) by closing stdin, if it is the only input plugin running within Logstash. After all input plugins are properly shut down, Logstash terminates after successful processing of all events. This allows the controlling process to properly determine, if Logstash has finished its work.
A short analysis of the Logstash input plugins showed, that there are not a lot of other plugins in the Logstash eco system, which share this behavior. One of the remaining possibilities would be logstash-input-unix plugin, if it is run in
mode => "client". In this case, the sending party (controlling process) is also able to shut down the logstash-input-unix plugin. The logstash-input-unix plugin may be used several times for a single Logstash instance, which allows to use one Logstash instance for all tests, which greatly improves the total duration for the tests, as Logstash is very slow to start compared to the effective time used for the tests.I have put together some lines, which extend logstash-filter-verifier to allow the usage of the logstash-input-unix plugin as input and therefore allows to run the tests in parallel.
This may also help for #11.