Derecho job_card update: processor and thread mapping option of mpiexec runtime arguments

[jkbk2004]

Description

Following up with PR https://github.com/ufs-community/ufs-weather-model/pull/1836 Derecho job_card runs in the format of mpiexec -n @[UFS_TASKS] -ppn @[PPN] --hostfile $PBS_NODEFILE ./fv3.exe However, a desirable way is to clearly apply and specify -depth option for the thread: mpiexec -n @[TASKS] -ppn @[TPN] -depth @[THRD] ./fv3.exe

Solution

Check with system derecho hpc managers and system admins for the detail formats of job_card.

[jkbk2004]

@ulmononian @natalie-perlin @RatkoVasic-NOAA @zach1221 @FernandoAndrade-NOAA FYI

[zach1221]

@jkbk2004 I met with and talked to a UCAR system admin for Derecho. They looked at the current job_card configuration but said it fit their best practices for the most part. They had some recommendations including adding --cpu-bind depth to the mpiexec line, and setting OMP_NUM_THREADS=1. I tested this and didn't notice a dramatic difference however, I haven't compared many test run results yet.