search

DHI / modelskill

Compare results from MIKE and other simulations with measurements
https://dhi.github.io/modelskill
MIT License
33 stars 8 forks source link

Speed up test execution #268

Open ecomodeller opened 1 year ago

ecomodeller commented 1 year ago

Is is possible to speed up some of these slow tests, or mark them as slow and only run them in CI?

$ pytest --durations=10
===================================================================================
3.30s call     tests/test_metrics.py::test_pr
1.47s call     tests/test_multivariable_compare.py::test_mv_mm_taylor
1.41s call     tests/test_multimodelcompare.py::test_mm_taylor
1.16s setup    tests/test_multivariable_compare.py::test_mv_mm_taylor
1.15s call     tests/test_multimodelcompare.py::test_mm_scatter
0.87s call     tests/test_combine_comparers.py::test_concat_model_different_time
0.80s call     tests/test_comparer.py::test_minimal_plots
0.79s call     tests/test_multimodelcompare.py::test_custom_metric_skilltable_mm_scatter
0.73s call     tests/regression/test_regression_rose.py::test_wind_rose_image_identical
0.60s call     tests/test_multivariable_compare.py::test_mv_mm_scatter
jsmariegaard commented 1 year ago

I have been thinking the same - I would like the test suite to run faster

jsmariegaard commented 1 year ago

How about reducing the file size of some key test data?

ecomodeller commented 1 year ago

How about reducing the file size of some key test data?

Sure, the files should as small as possible but no smaller🙄

ecomodeller commented 1 year ago

For now I think I will use https://testmon.org/ to only run tests that are affected by a code change to optimize the developer inner loop and then accept that it takes a few minutes to run all tests in CI.

jsmariegaard commented 11 months ago

I am getting these:

================================================ slowest 10 durations =================================================
3.14s call     tests/test_multimodelcompare.py::test_mm_scatter
1.19s call     tests/test_comparer.py::test_minimal_plots
0.92s call     tests/test_multimodelcompare.py::test_mm_taylor
0.91s call     tests/test_combine_comparers.py::test_concat_model_different_time
0.86s call     tests/test_multivariable_compare.py::test_mv_mm_scatter
0.64s call     tests/test_trackcompare.py::test_hist
0.64s setup    tests/test_combine_comparers.py::test_concat_model_different_time
0.59s call     tests/regression/test_regression_rose.py::test_wind_rose_image_identical
0.53s call     tests/test_multivariable_compare.py::test_mv_mm_taylor
0.49s call     tests/test_multimodelcompare.py::test_custom_metric_skilltable_mm_scatter