Reusing test suite - compare with baseline passes even if model does not run correctly

phil-blain / CICE

Development repository for the CICE sea-ice model

Other

0 stars 0 forks source link

Reusing test suite - compare with baseline passes even if model does not run correctly #1

Closed phil-blain closed 2 years ago

phil-blain commented 5 years ago

Happens because restarts from previous successful runs of the suite are present in the runs/test-name/restart directories.

phil-blain commented 5 years ago

Example:

$ cat test_output 
#---
COPY cesium_intel_smoke_gx3_1x1_diag1_diagimp_dynanderson_incremental_run3dt build
FAIL cesium_intel_smoke_gx3_1x1_diag1_diagimp_dynanderson_incremental_run3dt run -1 -1 -1
FAIL cesium_intel_smoke_gx3_1x1_diag1_diagimp_dynanderson_incremental_run3dt test 
PASS cesium_intel_smoke_gx3_1x1_diag1_diagimp_dynanderson_incremental_run3dt compare implicit-20190315T094936 6.02 5.16 0.58

phil-blain commented 5 years ago

maybe test script should abort if model aborts

phil-blain commented 2 years ago

reported in https://github.com/CICE-Consortium/CICE/issues/608 and fixed in https://github.com/CICE-Consortium/CICE/commit/85531cf8f8ae93f628344df6ecfa4be12e9629d8 for smoke and other tests.