[CI/CD] Run performance tests as a separate stage

functime-org / functime

Time-series machine learning at scale. Built with Polars for embarrassingly parallel feature extraction and forecasts on panel data.

https://docs.functime.ai

Apache License 2.0

1.02k stars 55 forks source link

[CI/CD] Run performance tests as a separate stage #143

Closed baggiponte closed 9 months ago

baggiponte commented 9 months ago

Running tests in CI always take a ton of time because we also run pytest-benchmark tests. To have a faster feedback loop, we might split those and run performance tests as a separate step. @ngriffiths13 proposed we use codspeed. What are the options? I would run those before merging every PR to ensure that we monitor for performance regressions too.

topher-lo commented 9 months ago

I believe benchmarks are disabled https://github.com/functime-org/functime/blob/main/.github/workflows/test.yml

Tests just take a long time right now with 2-core instances.

topher-lo commented 9 months ago

One thing to note about the test workflow trigger. It only triggers when a review is requested.