search

ComputationalCryoEM / ASPIRE-Python

Algorithms for Single Particle Reconstruction
http://spr.math.princeton.edu
GNU General Public License v3.0
49 stars 21 forks source link

Unit Tests Speed #746

Closed garrettwrong closed 1 year ago

garrettwrong commented 2 years ago

Locally they run reasonably fast (10-15m sequentially), but our Azure tests seem to take 30-60 minutes. If the total azure test takes over 60 minutes, the job is killed and fails CI.

We should

Regarding the second point, I placed example py37 timings from my macbook below. These are from develop at c3b13f4a. I filtered just the tests executing over one second. It's actually not as bad as I thought... so we might need to focus a little more on the platform/setup.

71.44s call     tests/test_apple.py::ApplePickerTestCase::testFileCorruption
58.34s call     tests/test_apple.py::ApplePickerTestCase::testEmptyCenters
49.57s call     tests/test_apple.py::ApplePickerTestCase::testPickCenters
30.75s call     tests/test_covar2d_denoiser.py::BatchedRotCov2DTestCase::testMSE
28.75s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testEstimateRotations_0
27.00s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testEstimateRotations_1
21.02s call     tests/test_CtfEstimate.py::CtfEstimatorTestCase::testEstimateCTF
20.28s call     tests/test_class2D.py::RIRClass2DTestCase::testRIRLegacy
14.15s call     tests/test_class2D.py::RIRClass2DTestCase::testRIRsk
13.07s call     tests/test_class2D.py::RIRClass2DTestCase::testRIRDevelBisp
9.82s call     tests/test_averager2d.py::ReddyChatterjiAverager2DTestCase::test_shifts_estimate
9.56s call     tests/test_averager2d.py::ReddyChatterjiAverager2DTestCase::test_rotations_estimate
8.78s call     tests/test_averager2d.py::ReddyChatterjiAverager2DTestCase::test_correlations_estimate
8.69s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testGlobalJSync
8.13s call     tests/test_covar3d.py::Covar3DTestCase::testCovar3D1
7.96s call     tests/test_covar3d.py::Covar3DTestCase::testCovar3D2
7.77s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testRelativeViewingDirections_0
7.47s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testRelativeViewingDirections_1
6.58s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testRelativeRotations_0
5.91s call     tests/test_averager2d.py::BFSRAverager2DTestCase::test_rotations_estimate
5.32s call     tests/test_CtfEstimate.py::CtfEstimatorTestCase::testRectangularMicrograph_1
5.15s call     tests/test_averager2d.py::BFSRAverager2DTestCase::test_shifts_estimate
5.13s call     tests/test_averager2d.py::BFSRAverager2DTestCase::test_correlations_estimate
5.01s call     tests/test_CtfEstimate.py::CtfEstimatorTestCase::testRectangularMicrograph_0
4.82s call     tests/test_adaptive_support.py::AdaptiveSupportTest::test_adaptive_support_F
4.68s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testCommonLines_1
4.45s call     tests/test_class2D.py::RIRClass2DTestCase::testSelectionImplementations
4.28s call     tests/test_preprocess_pipeline.py::PreprocessPLTestCase::testWhiten
4.27s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testCommonLines_0
4.22s call     tests/test_preprocess_pipeline.py::PreprocessPLTestCase::testWhiten2
4.21s call     tests/test_preprocess_pipeline.py::PreprocessPLTestCase::testInvertContrast
4.00s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testRelativeRotations_1
3.92s call     tests/test_volume.py::VolumeTestCase::testRotate_1
3.66s call     tests/test_image.py::ImageTestCase::testImShift
3.47s call     tests/test_simulation.py::SimTestCase::testSimulationSaveFile
3.00s call     tests/test_averager2d.py::BFSRAverager2DTestCase::testAverager
2.75s call     tests/test_coordinate_source.py::CoordinateSourceTestCase::testCommand
2.56s call     tests/test_bispectrum.py::BispectrumTestCase::testRotationalInvarianceFSPCA
2.40s call     tests/test_preprocess_pipeline.py::PreprocessPLTestCase::testPhaseFlip
2.26s call     tests/test_preprocess_pipeline.py::PreprocessPLTestCase::testNormBackground
2.20s call     tests/test_preprocess_pipeline.py::PreprocessPLTestCase::testEmptyPhaseFlip
2.19s call     tests/test_coordinate_source.py::CoordinateSourceTestCase::testEvenOddResize
2.15s call     tests/test_fft.py::ConfigTest::testCenteredFftn
2.10s call     tests/test_volume.py::VolumeTestCase::testCnSymmetricVolume
1.96s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testSelfCommonLines_0
1.87s call     tests/test_simulation.py::SimTestCase::testSimulationCached
1.70s call     tests/test_class2D.py::RIRClass2DTestCase::testSourceTooSmall
1.70s call     tests/test_class2D.py::RIRClass2DTestCase::testIncorrectSelectorClass
1.65s call     tests/test_orient_sync.py::OrientSyncTestCase::testEstShifts
1.58s call     tests/test_FPSWFbasis2D.py::FPSWFBasis2DTestCase::testEvaluate_t
1.46s call     tests/test_downsample.py::DownsampleTestCase::testDownsample3DCase_0
1.43s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testSelfRelativeRotations_1
1.42s call     tests/test_orient_sync.py::OrientSyncTestCase::testEstRotations
1.38s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testSelfCommonLines_1
1.36s call     tests/test_orient_symmetric.py::OrientSymmTestCase::testSelfRelativeRotations_0
1.36s call     tests/test_orient_sync.py::OrientSyncTestCase::testSyncMatrixVote
1.28s call     tests/test_fft.py::ConfigTest::testCenteredFft2
1.27s call     tests/test_class2D.py::RIRClass2DTestCase::testComponentSize
1.21s call     tests/test_coordinate_source.py::CoordinateSourceTestCase::testPreprocessing
1.13s call     tests/test_FBbasis2D.py::FBBasis2DTestCase_4::testGaussianExpand
1.10s call     tests/test_class2D.py::RIRClass2DTestCase::testEigenImages
1.08s call     tests/test_class2D.py::RIRClass2DTestCase::testIncorrectComponents
1.05s call     tests/test_class2D.py::RIRClass2DTestCase::testImplementations
1.03s call     tests/test_orient_sync.py::OrientSyncTestCase::testBuildCLmatrix
1.00s call     tests/test_averager2d.py::BFRAverager2DTestCase::test_shifts_estimate
garrettwrong commented 2 years ago

Perhaps the longest running one, tests/test_apple.py::ApplePickerTestCase::testFileCorruption could be corrupted sooner?

garrettwrong commented 1 year ago

Closed by several patches on 11/29. Disabling multiprocessing for Win and OSX on Azure, reducing pytest multiprocessing pool size, and patching a bug wrt pool launching.