search

esmf-org / esmf

The Earth System Modeling Framework (ESMF) is a suite of software tools for developing high-performance, multi-component Earth science modeling applications.
https://earthsystemmodeling.org/
Other
156 stars 75 forks source link

Sporadic failures in ESMF_CompTunnelUTest and ESMF_CompTunnelEx on Mac systems #237

Closed billsacks closed 5 months ago

billsacks commented 5 months ago

I have noticed sporadic failures in ESMF_CompTunnelUTest and ESMF_CompTunnelEx on my Mac (green). I have seen this when running multiple test suites at once, and have seen this at least once in nightly cron-based testing.

I first wondered if this had something to do with timeouts. @theurich says that these tests do have timeout values, but they're pretty large, so this is unlikely to be a performance-related issue.

@theurich says that what makes these tests unique is that they set up ports / sockets to make connections. If the port is used, then you have a problem – e.g., due to a hanging process. This can also come up if you run multiple instances of ESMF tests at the same time. Or a process from before could have crashed and there could be some hanging stuff. We have seen this periodically on Mac systems.

For now we'll just note that this can sometimes be an issue but close it as a wontfix. If this becomes a frequent issue, we could consider introducing a flag for testing that disables anything that could fail when running simultaneous test suites; these are the only tests that @theurich can think of that have this issue, but there could be others in the future.

billsacks commented 5 months ago

As noted above, closing as a wontfix - I just opened this to document the issue.