search

magnusbaeck / logstash-filter-verifier

Apache License 2.0
192 stars 27 forks source link

Logstash Config Test Coverage #95

Open breml opened 3 years ago

breml commented 3 years ago

If one has larger and more complex Logstash configuration (especially if there is some use of conditions), it gets harder and harder to make sure, one does have a test case for every branch of the configuration and the necessary coverage of the corner cases.

Therefore it would be beneficial, if Logstash Filter Verifier (LFV) would provide some support to track the "code" coverage of the Logstash configuration. The idea here is to use the Logstash pipeline stats (https://www.elastic.co/guide/en/logstash/current/node-stats-api.html#pipeline-stats) after the execution of the test cases to get the details, which filters have been hit. Given every filter has an ID configured, it would be possible to emit the name of the filters, which have not been hit by any test case.

magnusbaeck commented 3 years ago

It shouldn't be too hard to generate an HTML file with per-file stats and the configuration under test with non-covered filters marked with a different background color (similar to the output from e.g. gocov-html).

breml commented 3 years ago

It shouldn't be too hard to generate an HTML file with per-file stats and the configuration under test with non-covered filters marked with a different background color (similar to the output from e.g. gocov-html).

This is a nice idea. For a first implementation I would focus on the simpler thing and then improve from there.