search

ai2cm / pace

Implementation of the FV3GFS / SHiELD atmospheric model in Python
https://ai2cm.github.io/pace/
Apache License 2.0
37 stars 12 forks source link

Update gt4py on August 19, 2022 #285

Closed jdahm closed 2 years ago

jdahm commented 2 years ago

Purpose

Update gt4py to the gridtools/gt4py:master e61c49a.

jdahm commented 2 years ago

launch jenkins

jdahm commented 2 years ago

Hmm, first it's

E   cupy_backends.cuda.api.runtime.CUDARuntimeError: cudaErrorDevicesUnavailable: all CUDA-capable devices are busy or unavailable

now it's

12:13:25 E   ModuleNotFoundError: No module named 'xarray'
12:13:25 + '[' 4 -ne 0 ']'
12:13:25 + exitError 1510 73 'problem while executing script /tmp/workspace/pace-util_PR/slave/gce-1p100-v4/target/gpu/pace-util/.jenkins/actions/test.sh'
12:13:25 + rm -f /tmp/tmp..4753
12:13:25 + echo 'ERROR 1510: problem while executing script /tmp/workspace/pace-util_PR/slave/gce-1p100-v4/target/gpu/pace-util/.jenkins/actions/test.sh'
12:13:25 ERROR 1510: problem while executing script /tmp/workspace/pace-util_PR/slave/gce-1p100-v4/target/gpu/pace-util/.jenkins/actions/test.sh
12:13:25 + echo 'ERROR     LOCATION=.jenkins/jenkins.sh'
12:13:25 ERROR     LOCATION=.jenkins/jenkins.sh
12:13:25 + echo 'ERROR     LINE=73'
12:13:25 ERROR     LINE=73
12:13:25 + exit 1510

Thanks, daint!

jdahm commented 2 years ago

launch jenkins

jdahm commented 2 years ago

launch jenkins

jdahm commented 2 years ago

Jenkins - please test the correct branch. This is using the branch from another PR...

jdahm commented 2 years ago

launch jenkins

jdahm commented 2 years ago

re-launched Jenkins because the last run errored on the 54 rank test with

MPICH2 ERROR [Rank 48] [job id 40795605] [Tue Aug 23 18:42:08 2022] [c5-2c2s9n1] [nid04965] - MPID_nem_gni_check_localCQ(): GNI_CQ_EVENT_TYPE_POST had error (SOURCE_SSID:AT_MDD_INV:CPLTN_DREQ)

FlorianDeconinck commented 2 years ago

That MPI error is definitely jenkins being jenkins