implement testing framework based on py.test

DrSAR commented 8 years ago

Recent changes (#359, or 2to3 conversion) to the module(s) was terribly slow due to very difficult testing. In fact, we are still somewhat unclear which changes are working and which are.

Furthermore, a decent set of tests documents clearly the functionality of code (sometimes even before that code is written...). There might have been a previous issue like this but I can't find it.

I suggest we implement this using py.test since it's apparently the least invasive form of writing tests. This issue is supposed to only lay the groundwork and will require ongoing maintenance as new code is written.

It is not clear how this can be done for more visual output from certain functions. See also Andrew's thoughts in #102 .

DrSAR commented 8 years ago

Best Practices for pytest are discussed in their manual and translated to us they mean:

get rid of the test directory in the root of the package and put the tests together with the code
tests should be names test_modulename.py where module name can be an approximation of whatever is being tested.
consider pytest-cov to give us an idea of test coverage currently it's an abysmal 10%:

platform darwin -- Python 3.5.1, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /Users/stefan/sarpy, inifile: 
plugins: cov-2.2.1
collected 3 items 

sarpy/BRUKERIO/test_lowlevel.py ...
----------------------- coverage: platform darwin, python 3.5.1-final-0 ------------------------
Name                                          Stmts   Miss  Cover
-----------------------------------------------------------------
sarpy/BRUKERIO/AData_classes.py                 178    136    24%
sarpy/BRUKERIO/BRUKER_classes.py                530    427    19%
sarpy/BRUKERIO/JCAMP_comparison.py               40     30    25%
sarpy/BRUKERIO/JCAMP_dict.py                      1      1     0%
sarpy/BRUKERIO/SARlogger.py                      41     41     0%
sarpy/BRUKERIO/__init__.py                        0      0   100%
sarpy/BRUKERIO/bulk_analyze.py                  181    181     0%
sarpy/BRUKERIO/check_fid_acqp_method.py          18     18     0%
sarpy/BRUKERIO/compare_JCAMP_readers.py          48     48     0%
sarpy/BRUKERIO/congrid.py                        57     57     0%
sarpy/BRUKERIO/descriptortest.py                 63     63     0%
sarpy/BRUKERIO/lazy_property.py                  11      5    55%
sarpy/BRUKERIO/lowlevel.py                      362    177    51%
sarpy/BRUKERIO/masterlist.py                    101    101     0%
sarpy/BRUKERIO/mriBoards.py                     242    242     0%
sarpy/BRUKERIO/obj2dict.py                       30     30     0%
sarpy/BRUKERIO/oldtest_BRUKER_classes.py         25     25     0%
sarpy/BRUKERIO/oldtest_SARlogger.py              20     20     0%
sarpy/BRUKERIO/oldtest_lowlevel.py               71     71     0%
sarpy/BRUKERIO/oldtest_write2nii.py               8      8     0%
sarpy/BRUKERIO/parseJCAMP.py                     38     38     0%
sarpy/BRUKERIO/rois.py                          113    113     0%
sarpy/BRUKERIO/test_lowlevel.py                  21      3    86%
sarpy/BRUKERIO/visu_pars_2_Nifti1Header.py      100     92     8%
sarpy/BRUKERIO/write_csv.py                     165    165     0%
sarpy/BRUKERIO/write_parameter_stack.py          42     42     0%
sarpy/DCE/__init__.py                             0      0   100%
sarpy/DCE/tammo_lib.py                          283    283     0%
sarpy/ImageProcessing/__init__.py                 0      0   100%
sarpy/ImageProcessing/resample_onto.py           66     59    11%
sarpy/PKanalysis/PKModels.py                     54     54     0%
sarpy/PKanalysis/__init__.py                      0      0   100%
sarpy/__init__.py                                10      0   100%
sarpy/analysis/__init__.py                        0      0   100%
sarpy/analysis/gather_data.py                    41     41     0%
sarpy/analysis/get_histogram.py                  30     30     0%
sarpy/analysis/special_finder.py                 42     42     0%
sarpy/andrewcyung/BlochSiegert.py               203    203     0%
sarpy/andrewcyung/__init__.py                     0      0   100%
sarpy/fmoosvi/PKlib.py                          178    178     0%
sarpy/fmoosvi/VTC_test.py                         8      0   100%
sarpy/fmoosvi/__init__.py                         0      0   100%
sarpy/fmoosvi/analysis.py                      1008    959     5%
sarpy/fmoosvi/colormaps.py                       13     13     0%
sarpy/fmoosvi/getters.py                        177    156    12%
sarpy/fmoosvi/roipoly.py                         84     84     0%
sarpy/fmoosvi/test_suite.py                       0      0   100%
sarpy/fmoosvi/wrappers.py                       108     92    15%
sarpy/generate_summary.py                        97     97     0%
sarpy/helpers.py                                 60     30    50%
sarpy/qtsarpy/test_pyside.py                      9      6    33%
-----------------------------------------------------------------
TOTAL                                          4977   4461    10%

=================================== 3 passed in 5.46 seconds ===================================

DrSAR commented 8 years ago

This needs improvement and hence should be re-opened:

Travic-CI implementation: This is currently done in the public sandbox repo
write some more tests - the current ones for BRUKERIO/lowlevel.py are dependent on lots of data in the data folder. This won't work well without massive work for setup and teardown in a remote testing environment like Travis-CI

DrSAR commented 8 years ago

Attached to the testing framework milestone. Let's focus in this issue on creating testing templates for the various parts of code of interest. The idea is to increase test coverage and fill in with meaningful tests as we go along.

Issues should be created and added to the milestone for tests that require fleshing out.

DrSAR / SARlabpy

implement testing framework based on py.test #360