Closed xylar closed 1 year ago
I ran the pr
suite, pointing at a baseline that didn't actually exist. I see:
ocean/planar/baroclinic_channel/10km/threads
* step: init
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:05
* step: 1thread
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:02
* step: 2thread
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: validate
execution: SUCCESS
runtime: 0:00:00
task execution: SUCCESS
baseline comp.: FAIL
task runtime: 0:00:09
ocean/planar/baroclinic_channel/10km/decomp
* step: init
already completed
* step: 4proc
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: 8proc
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: validate
execution: SUCCESS
runtime: 0:00:00
task execution: SUCCESS
baseline comp.: FAIL
task runtime: 0:00:03
ocean/planar/baroclinic_channel/10km/restart
* step: init
already completed
* step: full_run
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: restart_run
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: validate
execution: SUCCESS
runtime: 0:00:00
task execution: SUCCESS
baseline comp.: FAIL
task runtime: 0:00:03
ocean/planar/inertial_gravity_wave
* step: init_200km
execution: SUCCESS
runtime: 0:00:01
* step: forward_200km
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:02
* step: init_100km
execution: SUCCESS
runtime: 0:00:01
* step: forward_100km
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:07
* step: init_50km
execution: SUCCESS
runtime: 0:00:02
* step: forward_50km
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:16
* step: init_25km
execution: SUCCESS
runtime: 0:00:07
* step: forward_25km
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:23
* step: analysis
execution: SUCCESS
runtime: 0:00:00
task execution: SUCCESS
baseline comp.: FAIL
task runtime: 0:01:00
ocean/single_column/cvmix
* step: init
execution: SUCCESS
runtime: 0:00:01
* step: forward
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: viz
execution: SUCCESS
runtime: 0:00:00
task execution: SUCCESS
baseline comp.: FAIL
task runtime: 0:00:02
ocean/single_column/ideal_age
* step: init
execution: SUCCESS
runtime: 0:00:01
* step: forward
execution: SUCCESS
baseline comp.: FAIL
runtime: 0:00:01
* step: viz
execution: SUCCESS
runtime: 0:00:00
task execution: SUCCESS
baseline comp.: FAIL
task runtime: 0:00:02
Task Runtimes:
0:00:09 FAIL ocean/planar/baroclinic_channel/10km/threads
0:00:03 FAIL ocean/planar/baroclinic_channel/10km/decomp
0:00:03 FAIL ocean/planar/baroclinic_channel/10km/restart
0:01:00 FAIL ocean/planar/inertial_gravity_wave
0:00:02 FAIL ocean/single_column/cvmix
0:00:02 FAIL ocean/single_column/ideal_age
Total runtime: 0:01:19
FAIL: 6 tasks failed, see above.
This seems like the expected behavior.
I compared the results with another run of this branch and to all 6 tests passing (including the baseline comparison).
@cbegeman, this is ready to review.
@cbegeman, thanks for having a look and trying it out!
If baseline comparison fails, raise an error at the task level not at the step level so task execution can complete.
Checklist
Testing
comment in the PR documents testing used to verify the changes