unit tests

[geoffwoollard]

use pytest to perform unit tests.

• [x] preprocess
• [x] SVD
• [x] map to map
• [x] distribution to distribution
• [ ] testing notebooks (low priority). See here and here

see example of pytest the compSPI/simSPI repo https://github.com/compSPI/simSPI/blob/master/.github/workflows/test.yml

Suggestions

• perform tests on subset of ground truth and submission data (e.g. 100 gt maps, 10 submitted maps), so that test runs in 10s - 1 min.
• make use of data validators to pass/fail test

[geoffwoollard]

Installs take 3.5 min because of installs. Reduced to ~2.5 in with pip caching. https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/pull/23

[geoffwoollard]

@DavidHerreros please do implement this and open a PR to compare the time and see if it helps.

https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/pull/23#issuecomment-2185846120

[DavidHerreros]

@geoffwoollard

While working on caching the test files and python environment, I encountered and error related to dependency versions.

It seems that in the new GitHub workflows being executed while testing the new GitHub actions, Scipy 1.14.0 was being installed. It seems that this version is not compatible with the optimal transport package coming with the dependencies. However, the package is not limiting the Scipy version itself, leading to a runtime error.

In other runs of the GitHub actions, Scipy 1.13.1 was installed instead. Trying this other version fixes the issue, so I think it is a good time to manually limit the version range allowed for the packages downloaded when installing the project. I would not start with anything fancy (it is even possible to limit a package version depending on the python version itself from the toml file), probably just giving a maximum version will solve most of the installation cases.

I will start by limiting the Scipy version to scipy<=1.13.1 in the pyproject.toml file. To make the installation more robust, I would go through the GiHub Actions log files and check the version of the other packages that work fine, and add a similar restriction to them.

[geoffwoollard]

Good catch!

[geoffwoollard]

For testing the notebooks, it would be nice to have a test config and pass in paths (overwrite the interactive aspect).

[geoffwoollard]

Closing - continue discussion about testing plots in this issue: https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues/42