search

logstash-plugins / logstash-filter-grok

Grok plugin to parse unstructured (log) data into something structured.
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
Apache License 2.0
122 stars 97 forks source link

Test: tunings for green(er) ci #163

Closed kares closed 3 years ago

kares commented 3 years ago

timeout specs should now be more predictable, performance specs are still not that much useful on CI ...

NOTE: would still like to keep perf. specs around for comparison if I (ever) manage to finish the faster timeout library.

colinsurprenant commented 3 years ago

A performance spec is still not passing.

yaauie commented 3 years ago

Yeh, I've kicked the build several times, and it looks like we're consistently running at about 105% overhead (52% reduction in throughput) while tolerating 100% overhead (50% reduction in throughput).

colinsurprenant commented 3 years ago

And ... I agree with @kares on «performance specs are still not of much useful on CI» ... There can be so much performance variance on CI build jobs that to avoid false positives we have to provide wider thresholds and at some point the test becomes useless because it wont catch potentially significant variations. IMO such performance tests should not be run on CI but should be periodically run on stable hardware and result stored to see variations in time.

kares commented 3 years ago

have disabled performance specs completely with a note - just if someone comes around wondering to not blindly turn them on. I am hoping with "lighter" timeout support, we eventually can still enabled the % degradation specs on CI, if not on Travis than at least on Azure where the virtual machines should be relatively more "predictable".