search

NOAA-EMC / NCEPLIBS-bufr

The NCEPLIBS-bufr library contains routines and utilites for working with the WMO BUFR format.
Other
44 stars 19 forks source link

Simplify util tests #376

Closed edwardhartnett closed 1 year ago

edwardhartnett commented 1 year ago

The util tests can be considerably simplified. For example, we have test_binv.sh, which contains:

set -eu

cmd=$1
outfile=$2
reffile=$3

rc="-1"
$cmd > $outfile && diff -w $outfile $reffile
rc=${?}

exit $rc

This is then used in the cmake file like this:

# Test binv utility
add_test(NAME test_binv  
  COMMAND ${CMAKE_BINARY_DIR}/bin/test_binv.sh "${CMAKE_BINARY_DIR}/utils/binv testfiles/data/satwndbufr" "testrun/binv.out"  "testfiles/testoutput/binv.out"
  )

This could all be replaced by one line of script code which runs the binv test with the arguments. If any aspects of the shell script need to be set at configure time, then we use test_binv.sh.in.

The cmake file should run the binv test script without providing any test details, those can be in the script file.

We need to add tests to the binv test script. Want the build script to contain all the testing details, for simple debugging.

It would be good if this could be done soon, so that we can proceed to test all utilities more thoroughly.

jbathegit commented 1 year ago

A question, and assuming I'm understanding correctly that your intent here is to have one add_test() for each utility which does nothing more than run the script, and where the script itself then does one or more runs of the actual utility for various cases...

Will gcovr still properly track code coverage for all such utilities? In other words, if multiple runs of the utility are now being done inside of a single script kicked off by a single add_test() (vs. each run of the utility being kicked off by its own separate add_test()), will gcovr still track everything the same way?

edwardhartnett commented 1 year ago

Yes, gcovr somehow figures it all out and still works.

This is how we run tests in netcdf-c, netcdf-fortran, PIO, and many other projects.

edwardhartnett commented 1 year ago

I believe this is done so I will close this issue.

jbathegit commented 1 year ago

It's not quite done yet. I just finished the test updates for the remaining utilities (cmpbqm, readbp, and readmp) and will push up a new PR soon.

edwardhartnett commented 1 year ago

Press on!