search

blue-yonder / tsfresh

Automatic extraction of relevant features from time series:
http://tsfresh.readthedocs.io
MIT License
8.45k stars 1.21k forks source link

Dockerise testing for dev #1093

Closed Scott-Simmons closed 3 days ago

Scott-Simmons commented 1 week ago

Related: https://github.com/blue-yonder/tsfresh/pull/994.

Easiest to go commit by commit. Uses 3 "flows"

(1) Test locally on specific python version Is useful for quick iteration. (2) Test locally on the discoverable python versions Is a useful sanity check for ensuring new development works on multiple versions of python. (3) Test in dedicated testing environment with guaranteed availability of python versions Is a good final sanity check. Takes longer, but creates a reproducible environment to test off and is a good 'source of truth'.

1 Test locally on specific python version

Testing time for one version is ~1 min. Note this includes the speedup from cached packages

2 Test locally on the discoverable python versions

image

Testing time across versions 3.7-3.11 is ~4 mins, but note that this speedup includes the speedup from cached packages.

Also note that python 3.7 fails on my machine due to some idiosyncratic issue related to my setup py37/lib/python3.7/site-packages/pip/_vendor/typing_extensions.py ... getting same error as here but not using vscode. Not an issue for me personally but shows an example of why testing within an isolated testing environment could be a good way to ensure reproducible version targeting.

3 Test in dedicated testing environment with guaranteed availability of python versions

image

Build time for testenv ~5 mins but this is a once-off build for the testenv.

Testing time across versions 3.7-->3.11 is ~6 mins.

Scott-Simmons commented 5 days ago

@nils-braun let me know If this is worth adding, hoping that it can helpful for the release process going forward