search

GeoscienceAustralia / ga_sar_workflow

InSAR processing workflow used by Geoscience Australia
Apache License 2.0
3 stars 1 forks source link

Coverage.py reports 0% statement execution with unit tested modules #114

Closed truth-quark closed 4 years ago

truth-quark commented 4 years ago

Coverage.py is reporting 0% statement execution for unit tested modules, both on Gadi and my local system.

The modules are being run, confirmed by adding raising an exception in the module code, which causes the tests to break. Below is an example of reproducing the issue on Gadi (coverage v4.5.4). 

$ hostname
gadi-login-06.gadi.nci.org.au
$ cd projects/gamma_insar/
$ git br
* pygamma_workflow
 gamma_insar]$ source configs/insar.env 
Loading pygamma-test-v1
  Loading requirement: udunits/2.2.26 dea-env/20191127 dea/20191127 fftw3/3.3.8 gtk/3.16.6 glib/2.36.4 gdk-pixbuf/2.28.2 hdf5/1.8.21 gmt/6.0.0
$
$ coverage --version
Coverage.py, version 4.5.4 with C extension
Documentation at https://coverage.readthedocs.io
$
$ coverage erase && coverage run --branch -m pytest -s tests/*.py && coverage report
/g/data/v10/public/modules/dea-env/20191127/lib/python3.6/site-packages/dash/testing/plugin.py:17: UserWarning:

run `pip install dash[testing]` if you need dash.testing

====================================== test session starts =================================================================
platform linux -- Python 3.6.7, pytest-5.3.1, py-1.8.0, pluggy-0.12.0
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/578/bpd578/projects/gamma_insar, inifile: setup.cfg
plugins: benchmark-3.2.3, openfiles-0.4.0, remotedata-0.3.1, astropy-header-0.1.1, hypothesis-4.47.3, doctestplus-0.4.0, celery-4.3.0, dash-1.4.1, arraydiff-0.3, cov-2.8.1
collected 25 items                                                                                                                                                                                        

tests/test_process_ifg.py .....
tests/test_project.py ............
tests/test_py_gamma_ga.py ........

Name                                    Stmts   Miss Branch BrPart     Cover
----------------------------------------------------------------------------
__init__.py                                 0      0      0      0   100.00%
conftest.py                                 3      0      0      0   100.00%
insar/__init__.py                           6      6      0      0     0.00%
insar/_version.py                         279    279    144      0     0.00%
insar/calc_baselines_new.py               350    350    102      0     0.00%
insar/calc_multilook_values.py             70     70      8      0     0.00%
insar/clean_up.py                          62     62     38      0     0.00%
insar/constant.py                         150    150      0      0     0.00%
insar/coreg_utils.py                       53     53     44      0     0.00%
insar/coregister_dem.py                   738    738    113      0     0.00%
insar/coregister_slc.py                   478    478     80      0     0.00%
insar/generate_slc_inputs.py               87     87     38      0     0.00%
insar/logs.py                              13     13      0      0     0.00%
insar/make_gamma_dem.py                    56     56      8      0     0.00%
insar/meta_data/__init__.py                 0      0      0      0   100.00%
insar/meta_data/grid_def_filter.py         27     27      6      0     0.00%
insar/meta_data/metadata_diagnosis.py     101    101     46      0     0.00%
insar/meta_data/s1_gridding_utils.py      226    226    133      0     0.00%
insar/meta_data/s1_slc.py                 721    721    336      0     0.00%
insar/process_ifg.py                       83     83     20      0     0.00% **ERROR: should not be zero!**
insar/process_s1_slc.py                   335    335     92      0     0.00%
insar/project.py                          186    186     24      0     0.00% **ERROR: should not be zero!**
insar/py_gamma_ga.py                       67     67     26      0     0.00% **ERROR: should not be zero!**
insar/s1_slc_metadata.py                  473    473    239      0     0.00%
insar/scripts/__init__.py                   0      0      0      0   100.00%
insar/scripts/grid_processing.py          218    218     58      0     0.00%
insar/scripts/insar_pbs.py                133    133     38      0     0.00%
insar/scripts/package_insar.py            189    189     54      0     0.00%
insar/scripts/process_gamma.py            434    434    130      0     0.00%
insar/subprocess_utils.py                  84     84     24      0     0.00%
insar/xml_util.py                          46     46      8      0     0.00%
setup.py                                   10     10      0      0     0.00%
tests/__init__.py                           7      0      2      1    88.89%
tests/test_process_ifg.py                 133      0      6      0   100.00%
tests/test_project.py                     160      0      6      0   100.00%
tests/test_py_gamma_ga.py                  64      1      4      0    98.53%
versioneer.py                             568    568    232      0     0.00%
----------------------------------------------------------------------------
TOTAL                                    6650   6284   2065      1     4.39%

Running the tests with a single module arg to pytest (pytest -m tests/test_process_ifg.py) also results in 0% coverage.

My local virtual environment has coverage 5.2.1 & exhibits the same behaviour.

Hypotheses as to the source of the problem:

jeremyh commented 4 years ago

I recommend using pytest-cov instead of coverage.py manually.

pip install it and see if this works better:

pytest --cov insar tests insar
truth-quark commented 4 years ago

Interesting... the coverage docs recommended using coverage manually. pytest-cov reports > 0% test coverage on both Gadi and my laptop, so it's doing something differently.

jeremyh commented 4 years ago

pytest does some magic bytecode rewriting, which might be messing it up... not sure!

truth-quark commented 4 years ago

Using pytest-cov solves this problem.