search

NOAA-EMC / NCEPLIBS-ip2

Other
1 stars 4 forks source link

Build tests with CMake #17

Closed kgerheiser closed 3 years ago

kgerheiser commented 3 years ago

I did some re-factoring and now I have most of the tests working in CMake. It tests various combinations of grid/interpolation types against a baseline. Still working on the vector tests.

Just have to run make test

Fixes #6

Fixes #12

Fixes #13

Fixes #14

Might do some re-factoring on how I handle the tests in CMake.

What do you think @edwardhartnett

GeorgeGayno-NOAA commented 3 years ago

The tests have several 'readme' files, which appear to be out-of-date.

kgerheiser commented 3 years ago

Yes, everything is correct. I am not finished yet. I will remove/update the test readme and the unused things at the end.

kgerheiser commented 3 years ago

Can't seem to get the vector tests to work. I did everything just like I did for the scalar tests and the code is mostly the same.

The input data looks fine, the output data doesn't look crazy (reasonable values, no NaNs, inf, etc) except it doesn't match the baseline.

Any ideas @GeorgeGayno-NOAA 

32-bit gcc

        360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file = ./data/baseline_data/vector/4_byte_bin/grid3.opt0.bin_4
 - MAX/MIN OF U-WIND DATA:    59.5100021      -26.6049995
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.9846497
 - MAX DIFFERENCE:    57.4199982
 - AVG DIFFERENCE:    9.49930286
 - MAX/MIN OF V-WIND DATA:    51.8300018      -52.0299988
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.9739075
 - MAX DIFFERENCE:    71.8700027
 - AVG DIFFERENCE:    10.9740810
 Expected 0 points > 0, found:       130293
ERROR STOP

64-bit gcc
        360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file = ./data/baseline_data/vector/8_byte_bin/grid3.opt0.bin_8
 - MAX/MIN OF U-WIND DATA:    59.510000228897923       -26.604999542239447
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.984653161448747
 - MAX DIFFERENCE:    57.419998168945312
 - AVG DIFFERENCE:    9.4993250900125012
 - MAX/MIN OF V-WIND DATA:    51.829999923866524       -52.029998779155875
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.973910374462861
 - MAX DIFFERENCE:    71.870002746582031
 - AVG DIFFERENCE:    10.974111647505033
 Expected 0 points > 0, found:       130293
ERROR STOP

64-bit intel
tests$ ./test_vector 3 0
         360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file =
 ./data/baseline_data/vector/8_byte_bin/grid3.opt0.bin_8

 - MAX/MIN OF U-WIND DATA:    59.5100002288933       -26.6049995423189
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.9846531614487
 - MAX DIFFERENCE:    57.4199981689453
 - AVG DIFFERENCE:    9.49932508627886
 - MAX/MIN OF V-WIND DATA:    51.8299999237166       -52.0299987793838
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.9739103744629
 - MAX DIFFERENCE:    71.8700027465820
 - AVG DIFFERENCE:    10.9741116437811
 Expected 0 points > 0, found:       130293
GeorgeGayno-NOAA commented 3 years ago

Can't seem to get the vector tests to work. I did everything just like I did for the scalar tests and the code is mostly the same.

The input data looks fine, the output data doesn't look crazy (reasonable values, no NaNs, inf, etc) except it doesn't match the baseline.

Any ideas @GeorgeGayno-NOAA

32-bit gcc

        360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file = ./data/baseline_data/vector/4_byte_bin/grid3.opt0.bin_4
 - MAX/MIN OF U-WIND DATA:    59.5100021      -26.6049995
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.9846497
 - MAX DIFFERENCE:    57.4199982
 - AVG DIFFERENCE:    9.49930286
 - MAX/MIN OF V-WIND DATA:    51.8300018      -52.0299988
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.9739075
 - MAX DIFFERENCE:    71.8700027
 - AVG DIFFERENCE:    10.9740810
 Expected 0 points > 0, found:       130293
ERROR STOP

64-bit gcc
        360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file = ./data/baseline_data/vector/8_byte_bin/grid3.opt0.bin_8
 - MAX/MIN OF U-WIND DATA:    59.510000228897923       -26.604999542239447
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.984653161448747
 - MAX DIFFERENCE:    57.419998168945312
 - AVG DIFFERENCE:    9.4993250900125012
 - MAX/MIN OF V-WIND DATA:    51.829999923866524       -52.029998779155875
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.973910374462861
 - MAX DIFFERENCE:    71.870002746582031
 - AVG DIFFERENCE:    10.974111647505033
 Expected 0 points > 0, found:       130293
ERROR STOP

64-bit intel
tests$ ./test_vector 3 0
         360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file =
 ./data/baseline_data/vector/8_byte_bin/grid3.opt0.bin_8

 - MAX/MIN OF U-WIND DATA:    59.5100002288933       -26.6049995423189
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.9846531614487
 - MAX DIFFERENCE:    57.4199981689453
 - AVG DIFFERENCE:    9.49932508627886
 - MAX/MIN OF V-WIND DATA:    51.8299999237166       -52.0299987793838
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.9739103744629
 - MAX DIFFERENCE:    71.8700027465820
 - AVG DIFFERENCE:    10.9741116437811
 Expected 0 points > 0, found:       130293

@kgerheiser are all the vector tests failing? or only the grid 3, interp option 0 tests? Are you comparing the output to the wrong baseline? I can look at the output. Where are you running the tests?

kgerheiser commented 3 years ago

All the vector tests fail. I picked 3 0 because that seems simple and even with the scalars I could get bitwise-identical results with real32 (some tests don't).

I think I'm comparing to the correct baseline. I did everything the same as I did with the scalar tests. (It prints out the baseline file in the log)

I ran the tests on my local machine and on Hera, and they produce the same, if not correct results.

You can checkout my branch, load sp, cmake .., make, and make test Or cd tests, ./test_vector <grid> <interp> if you want to do it manually.

GeorgeGayno-NOAA commented 3 years ago

All the vector tests fail. I picked 3 0 because that seems simple and even with the scalars I could get bitwise-identical results with real32 (some tests don't).

I think I'm comparing to the correct baseline. I did everything the same as I did with the scalar tests. (It prints out the baseline file in the log)

I ran the tests on my local machine and on Hera, and they produce the same, if not correct results.

You can checkout my branch, load sp, cmake .., make, and make test Or cd tests, ./test_vector <grid> <interp> if you want to do it manually.

Will do. I want to learn to run the tests anyway.

GeorgeGayno-NOAA commented 3 years ago

All the vector tests fail. I picked 3 0 because that seems simple and even with the scalars I could get bitwise-identical results with real32 (some tests don't).

I think I'm comparing to the correct baseline. I did everything the same as I did with the scalar tests. (It prints out the baseline file in the log)

I ran the tests on my local machine and on Hera, and they produce the same, if not correct results.

You can checkout my branch, load sp, cmake .., make, and make test Or cd tests, ./test_vector <grid> <interp> if you want to do it manually.

I am able to run the tests on Hera. How do you look at the output? I don't see where the tests write out anything.

kgerheiser commented 3 years ago

To see stdout of the tests run ctest --verbose in place of make test in the build directory

The test executables are placed under build/tests

GeorgeGayno-NOAA commented 3 years ago

Can't seem to get the vector tests to work. I did everything just like I did for the scalar tests and the code is mostly the same.

The input data looks fine, the output data doesn't look crazy (reasonable values, no NaNs, inf, etc) except it doesn't match the baseline.

I plotted the output data, it looks like the poles are flipped and it is shifted 180 degrees east/west. Will dig deeper.

Any ideas @GeorgeGayno-NOAA

32-bit gcc

        360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file = ./data/baseline_data/vector/4_byte_bin/grid3.opt0.bin_4
 - MAX/MIN OF U-WIND DATA:    59.5100021      -26.6049995
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.9846497
 - MAX DIFFERENCE:    57.4199982
 - AVG DIFFERENCE:    9.49930286
 - MAX/MIN OF V-WIND DATA:    51.8300018      -52.0299988
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.9739075
 - MAX DIFFERENCE:    71.8700027
 - AVG DIFFERENCE:    10.9740810
 Expected 0 points > 0, found:       130293
ERROR STOP

64-bit gcc
        360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file = ./data/baseline_data/vector/8_byte_bin/grid3.opt0.bin_8
 - MAX/MIN OF U-WIND DATA:    59.510000228897923       -26.604999542239447
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.984653161448747
 - MAX DIFFERENCE:    57.419998168945312
 - AVG DIFFERENCE:    9.4993250900125012
 - MAX/MIN OF V-WIND DATA:    51.829999923866524       -52.029998779155875
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.973910374462861
 - MAX DIFFERENCE:    71.870002746582031
 - AVG DIFFERENCE:    10.974111647505033
 Expected 0 points > 0, found:       130293
ERROR STOP

64-bit intel
tests$ ./test_vector 3 0
         360         181
 - CALL IPOLATEV FOR GRID: 3
 - USE INTERP OPTION: 0
 baseline_file =
 ./data/baseline_data/vector/8_byte_bin/grid3.opt0.bin_8

 - MAX/MIN OF U-WIND DATA:    59.5100002288933       -26.6049995423189
 - NUMBER OF PTS DIFFERENT:        65150
 - PERCENT OF TOTAL:    99.9846531614487
 - MAX DIFFERENCE:    57.4199981689453
 - AVG DIFFERENCE:    9.49932508627886
 - MAX/MIN OF V-WIND DATA:    51.8299999237166       -52.0299987793838
 - NUMBER OF PTS DIFFERENT:        65143
 - PERCENT OF TOTAL:    99.9739103744629
 - MAX DIFFERENCE:    71.8700027465820
 - AVG DIFFERENCE:    10.9741116437811
 Expected 0 points > 0, found:       130293
GeorgeGayno-NOAA commented 3 years ago

@kgerheiser Check how you are setting "input_gdtmpl" (and associated variables) in routine interp_mod.f90. It does not look like you are setting it to "vector_input_gdtmpl" (which you define in input_data_mod.f90) for the vector test.

kgerheiser commented 3 years ago

Yay! That worked

Thanks George.

I'll clean it up a bit and make a real PR

kgerheiser commented 3 years ago

All the tests work now and I cleaned up the README. I haven't really looked into the reg_test directory. Is that something that should be ported over?

Either way, I'll do that in a separate PR and get this into develop first.

Ready for review @GeorgeGayno-NOAA, @edwardhartnett

GeorgeGayno-NOAA commented 3 years ago

All the tests work now and I cleaned up the README. I haven't really looked into the reg_test directory. Is that something that should be ported over?

Either way, I'll do that in a separate PR and get this into develop first.

Ready for review @GeorgeGayno-NOAA, @edwardhartnett

@kgerheiser We can discuss what to do with the 'reg tests'. They are more comprehensive than the unit tests. The latter were created well after the 'reg tests' to help with porting to new machines (since they compare to baseline data). The 'reg tests' were designed to compare two different tags of the library. (I am not sure I am even using the terms unit/regression test properly). You can merge what you have if you want.