Increase code coverage in testing

fmalatino commented 1 month ago

Is your feature request related to a problem? Please describe. Through the use of coverage current code coverage in test suite is 33%. Coverage of stencils should optimally be at ~75% Output from coverage:

Name                                                       Stmts   Miss Branch BrPart  Cover
--------------------------------------------------------------------------------------------
pyFV3/_config.py                                             281     42     84      2    80%
pyFV3/dycore_state.py                                        120     39     52      1    64%
pyFV3/initialization/analytic_init.py                         20     11      6      0    35%
pyFV3/initialization/init_utils.py                           139     92     12      0    31%
pyFV3/initialization/test_cases/initialize_baroclinic.py      89     67      4      0    24%
pyFV3/initialization/test_cases/initialize_tc.py             164    164      2      0     0%
pyFV3/stencils/a2b_ord4.py                                   252    125     98     12    47%
pyFV3/stencils/c_sw.py                                       174    120     68      3    24%
pyFV3/stencils/d2a2c_vect.py                                 252    136    112      8    39%
pyFV3/stencils/d_sw.py                                       323    132    140     18    51%
pyFV3/stencils/del2cubed.py                                   90     45     40      0    36%
pyFV3/stencils/delnflux.py                                   252    129    155      8    37%
pyFV3/stencils/divergence_damping.py                         201     68     74      7    57%
pyFV3/stencils/dyn_core.py                                   277    122    114     11    46%
pyFV3/stencils/fillz.py                                       78     57     50      0    21%
pyFV3/stencils/fv_dynamics.py                                188    145     80      0    18%
pyFV3/stencils/fv_subgridz.py                                377    331     90      0    14%
pyFV3/stencils/fvtp2d.py                                      83     16     24      2    74%
pyFV3/stencils/fxadv.py                                      141     81     66      3    30%
pyFV3/stencils/map_single.py                                  59     21     16      0    59%
pyFV3/stencils/mapn_tracer.py                                 24      1      8      2    91%
pyFV3/stencils/moist_cv.py                                    52     33     20      0    43%
pyFV3/stencils/neg_adj3.py                                   188    169    106      0     8%
pyFV3/stencils/nh_p_grad.py                                   51     22     10      0    48%
pyFV3/stencils/pe_halo.py                                     11      8     10      0    14%
pyFV3/stencils/pk3_halo.py                                    24     10     10      0    41%
pyFV3/stencils/ppm.py                                         40     26     20      0    30%
pyFV3/stencils/ray_fast.py                                    94     67     84      0    21%
pyFV3/stencils/remap_profile.py                              318    276    220      2    11%
pyFV3/stencils/remapping.py                                  173     77     88      8    41%
pyFV3/stencils/riem_solver3.py                                74     45     30      1    31%
pyFV3/stencils/riem_solver_c.py                               57     31     28      0    33%
pyFV3/stencils/saturation_adjustment.py                      525    400    200      1    30%
pyFV3/stencils/sim1_solver.py                                 77     64     58      0    11%
pyFV3/stencils/temperature_adjust.py                          18     13     10      0    18%
pyFV3/stencils/tracer_2d_1l.py                                94     71     36      0    21%
pyFV3/stencils/updatedzc.py                                   72     36     28      1    45%
pyFV3/stencils/updatedzd.py                                   84     31     24      1    54%
pyFV3/stencils/xppm.py                                       169    122     78      3    32%
pyFV3/stencils/xtp_u.py                                       35     27     16      0    24%
pyFV3/stencils/yppm.py                                       170    122     78      3    33%
pyFV3/stencils/ytp_v.py                                       35     27     16      0    24%
pyFV3/testing/map_single.py                                   15     15      2      0     0%
pyFV3/testing/translate_dyncore.py                            55      3     16      1    94%
pyFV3/testing/translate_fvdynamics.py                         81     52     24      0    30%
pyFV3/testing/validation.py                                   89     62     20      0    27%
pyFV3/utils/functional_validation.py                          31     25      4      0    17%
pyFV3/version.py                                               2      0      0      0   100%
pyFV3/wrappers/geos_wrapper.py                               275    275     32      0     0%
--------------------------------------------------------------------------------------------
TOTAL                                                       6493   4053   2563     98    33%

Describe the solution you'd like Additional tests to increase coverage

FlorianDeconinck commented 1 month ago

Those results are surprising, considering that they were generated with Acoustics which touches upon most of the code. The next steps are:

Run FVDynamics which is exactly all the dycore
Dive in some of the more central test and understand why the discrepancy (could be configuration, lack of data, etc.)

Ultimately, we should tie a coverage to a dataset so we now how to reach global coverage.

FlorianDeconinck commented 3 weeks ago

Thinking more about this, it actually makes sense.

The way the code works we never "execute" the stencil code declared under gt4py frontend. Instead we code generate then we jump over to the generate code in the .gt_cache_X folders. This means the coverage will always see those code as "uncovered".

Partial fix is to include in the coverage the cache directory. Partial solution, because we still want the coverage to look at what code is being touched overall in-between stencils. Also __INLINED and other code culling system within the stack will chop away code and will remove use from being able to have coverage look at them. Maybe this can only be fixed by custom implementation of coverage tool from with gt4py and/or orchestration.

This issue will be worse under orchestration.

NOAA-GFDL / PyFV3

Increase code coverage in testing #22