thanos-io / promql-engine

Multi-threaded PromQL engine implementation based on the Volcano paper.
Apache License 2.0
133 stars 52 forks source link

Improve test suite speed #412

Closed pedro-stanaka closed 3 months ago

pedro-stanaka commented 3 months ago

Summary

Using t.Parallel to speed up tests and also creating a faster make target to test locally (running all tests, but using build caches). Also, pulling some of data creation on tests outside so we can avoid generating a lot of data without reason.

Having faster test suit improves feedback loop and CI times.

The native histogram tests are still taking an awful amount of time, but I did not had time to take a look into extracting data creation.

On main I get:

      329.71 real       411.05 user        37.16 sys

On my branch I get:

       80.68 real       434.55 user        90.90 sys

without racing it is a much better:

real    0m13.486s    user    0m48.597s   sys     0m4.594s

Changes

MichaHoffmann commented 3 months ago

Awesome, this is great! Thanks

$ time go test .
ok      github.com/thanos-io/promql-engine/engine   30.370s

This usually takes 120s on main for me