NISOx-BDI / SwE-toolbox-testdata

This repository contains test data for the SwE-toolbox CI tests.
1 stars 2 forks source link

The current computation time of the tests is relatively slow #13

Closed BryanGuillaume closed 4 years ago

BryanGuillaume commented 4 years ago

One issue of the current tests is that they can take quite a long time, preventing a fast iteration between testing and code modification. Moreover, this somehow discourages us to add more tests that seem needed as many options/scenarios are not currently tested at all.

Thus, in order to reduce this computation time while not modifying too much the current tests, I would like to suggest to reduce the data analysed by simply downsampling the current test dataset. To evaluate the effect of this downsampling on the computation time, I have run a small experiment where I created two other datasets from the original with voxel sizes of 6x6x6 mm and 9x9x9 mm instead of 3x3x3. Here below is a table with some average computation times on my computer:

Voxel Size In-mask voxels Avg. computation time for a vox. param. test Avg. computation time for a vox. WB (nB = 20) test
3x3x3 51,560 52'' 684''
6x6x6 6,591 20'' 115''
9x9x9 2,325 18'' 50''

It seems that the gain from using 6x6x6 instead of 3x3x3 is clearly beneficial. The gain from using 9x9x9 instead of 6x6x6 is also beneficial but not by much for parametric analyses.

As the number of in-mask 9x9x9 voxels seems still reasonable (more than 2K), I would suggest replacing the original test dataset by this 9x9x9 dataset for all the tests. We could expect a speedup by a factor of 10 as the majority of the tests are actually WB tests.

@nicholst, are you ok with this change?

nicholst commented 4 years ago

Yes! Sounds good! --


Thomas Nichols, PhD Professor of Neuroimaging Statistics Nuffield Department of Population Health | University of Oxford Big Data Institute | Li Ka Shing Centre for Health Information and Discovery Old Road Campus | Headington | Oxford | OX3 7LF | United Kingdom T: +44 1865 743590 | E: thomas.nichols@bdi.ox.ac.uk W: http://nisox.org | http://www.bdi.ox.ac.uk

BryanGuillaume commented 4 years ago

This issue was addressed with PR #14.