search

esmf-org / esmf

The Earth System Modeling Framework (ESMF) is a suite of software tools for developing high-performance, multi-component Earth science modeling applications.
https://earthsystemmodeling.org/
Other
156 stars 75 forks source link

Add an env var to allow skipping performance tests #196

Closed billsacks closed 10 months ago

billsacks commented 10 months ago

Add a new environment variable ESMF_TESTPERFORMANCE. If set to "OFF", this will skip checking performance in performance tests.

This is useful on machines where performance tests are highly variable and can lead to spurious failures. e.g., this happens on green (a MacOS / Darwin system), where the nightly cron job can run very slowly, leading to failures in some performance tests (generally in ESMF_ArrayRedistPerfUTest).

See https://github.com/esmf-org/esmf-test-scripts/commit/229f023a9e1ad25a40715be23272c110efc78b82 for how I am using this for nightly testing.

billsacks commented 10 months ago

@theurich can you please give this a quick look whenever you get a chance and let me know if this implementation is in line with what you imagined?