NOAA-GFDL / fre-cli

Python-based command line interface for FRE (FMS Runtime Environment) to compile and run FMS-based models and post-process their output.
GNU Lesser General Public License v3.0
3 stars 7 forks source link

include pytest-cov meta.yaml #126

Closed ilaflott closed 1 month ago

ilaflott commented 1 month ago

Describe your changes

add pytest-cov coverage tool to pytest calls in meta.yaml, see how well it works.

not totally sure I want to merge this yet, but it's easier to run the workflow with a draft PR open.

Issue ticket number and link (if applicable)

Issue #122

Checklist before requesting a review

ilaflott commented 1 month ago

Taking out of draft- the workflow will not run for draft PRs

ilaflott commented 1 month ago

Seems to work great!

Name                                                                  Stmts   Miss  Cover
-----------------------------------------------------------------------------------------
fre/__init__.py                                                          10     10     0%
fre/app/__init__.py                                                       3      3     0%
fre/app/freapp.py                                                        29     29     0%
fre/app/generate_time_averages/__init__.py                                1      1     0%
fre/app/generate_time_averages/cdoTimeAverager.py                        53     53     0%
fre/app/generate_time_averages/frenctoolsTimeAverager.py                 33     33     0%
fre/app/generate_time_averages/frepytoolsTimeAverager.py                130    130     0%
fre/app/generate_time_averages/generate_time_averages.py                 33     33     0%
fre/app/generate_time_averages/tests/test_generate_time_averages.py     139     55    60%
fre/app/generate_time_averages/timeAverager.py                           32     32     0%
fre/app/maskAtmosPlevel.py                                               77     77     0%
fre/catalog/__init__.py                                                   2      2     0%
fre/catalog/frecatalog.py                                                19     19     0%
fre/catalog/gen_intake_gfdl.py                                           42     42     0%
fre/check/__init__.py                                                     3      3     0%
fre/check/frecheck.py                                                    12     12     0%
fre/check/frecheckexample.py                                              9      9     0%
fre/cmor/CMORmixer.py                                                   199    199     0%
fre/cmor/__init__.py                                                      3      3     0%
fre/cmor/frecmor.py                                                      16     16     0%
fre/fre.py                                                                8      8     0%
fre/lazy_group.py                                                        22     22     0%
fre/list/__init__.py                                                      3      3     0%
fre/list/frelist.py                                                      12     12     0%
fre/list/frelistexample.py                                                9      9     0%
fre/make/__init__.py                                                      7      7     0%
fre/make/createCheckout.py                                               55     55     0%
fre/make/createCompile.py                                                53     53     0%
fre/make/createDocker.py                                                 47     47     0%
fre/make/createMakefile.py                                               43     43     0%
fre/make/fremake.py                                                      66     66     0%
fre/make/runFremake.py                                                   91     91     0%
fre/pp/__init__.py                                                       10     10     0%
fre/pp/checkoutScript.py                                                 32     32     0%
fre/pp/configure_script_xml.py                                          288    288     0%
fre/pp/configure_script_yaml.py                                          87     87     0%
fre/pp/frepp.py                                                          86     86     0%
fre/pp/install.py                                                        10     10     0%
fre/pp/run.py                                                            10     10     0%
fre/pp/status.py                                                         10     10     0%
fre/pp/validate.py                                                       13     13     0%
fre/pp/wrapper.py                                                        55     55     0%
fre/run/__init__.py                                                       3      3     0%
fre/run/frerun.py                                                        12     12     0%
fre/run/frerunexample.py                                                  9      9     0%
fre/test/__init__.py                                                      3      3     0%
fre/test/fretest.py                                                      12     12     0%
fre/test/fretestexample.py                                                9      9     0%
fre/tests/test_fre_cli.py                                                12      0   100%
fre/yamltools/__init__.py                                                 3      3     0%
fre/yamltools/data_table/__init__.py                                      0      0   100%
fre/yamltools/data_table/combine_data_table_yamls.py                     45     45     0%
fre/yamltools/data_table/data_table_to_yaml.py                           72     72     0%
fre/yamltools/data_table/is_valid_data_table_yaml.py                     64     64     0%
fre/yamltools/diag_table/__init__.py                                      0      0   100%
fre/yamltools/diag_table/combine_diag_table_yamls.py                     86     86     0%
+ pylint fre/
fre/yamltools/diag_table/diag_table_to_yaml.py                          250    250     0%
fre/yamltools/diag_table/is_valid_diag_table_yaml.py                    141    141     0%
fre/yamltools/field_table/__init__.py                                     0      0   100%
fre/yamltools/field_table/combine_field_table_yamls.py                   75     75     0%
fre/yamltools/field_table/field_table_to_yaml.py                        177    177     0%
fre/yamltools/freyamltools.py                                            12     12     0%
fre/yamltools/freyamltoolsexample.py                                      9      9     0%
-----------------------------------------------------------------------------------------
TOTAL                                                                  2856   2760     3%
ilaflott commented 1 month ago

aacccctuualllllly i don't know why i'm getting coverage metrics for tests/... i assumed pytest scripts would be omitted.

ceblanton commented 1 month ago

I see what you mean. How does it distinguish the test scripts from the actual code, you're asking.

fre/tests/test_fre_cli.py should be counted as coverage for fre/fre.py right?

ilaflott commented 1 month ago

I see what you mean. How does it distinguish the test scripts from the actual code, you're asking.

fre/tests/test_fre_cli.py should be counted as coverage for fre/fre.py right?

latest commit should take care of this! see .coveragerc and .pytest.ini

ceblanton commented 1 month ago

can you see the updated coverage output in the CI logs? I looked for it but couldn't see it.

The documentation CI "test" is unhappy now, not related to these updates I think.

ilaflott commented 1 month ago

can you see the updated coverage output in the CI logs? I looked for it but couldn't see it.

... no, but i think i see something relevant here... from our CI/CD:

platform linux -- Python 3.9.19, pytest-7.4.4, pluggy-1.0.0
rootdir: $SRC_DIR
plugins: cov-4.1.0, anyio-4.2.0

... from my local copy however:

platform linux -- Python 3.9.19, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/Ian.Laflotte/Working/fre-cli_pytest_cov
configfile: .pytest.ini
plugins: cov-5.0.0, anyio-4.4.0
collected 20 items                                                                                                                                                                          

The documentation CI "test" is unhappy now, not related to these updates I think.

for doc changes that fix this: https://github.com/NOAA-GFDL/fre-cli/pull/125

ilaflott commented 1 month ago

looks good enough to me- the new empty __init.py__ files get caught by the coverage tool, but the testing code does not! additionally, the coverage metric isn't affected, since the added files are empty.

merging!