geodesymiami / rsmas_insar

RSMAS InSAR code
https://rsmas-insar.readthedocs.io/
GNU General Public License v3.0
59 stars 23 forks source link

submit_jobs.bash: need to adjust to qlimits format of frontera #488

Closed falkamelung closed 3 years ago

falkamelung commented 3 years ago

On frontera

MAX_JOBS_PER_QUEUE=$(qlimits | grep $QUEUENAME | awk '{print $4}')

does not work as the qlimits call is different. My understanding is that MaxJobsPU is the correct field (100 for flex)

We might have to check for platform as we do in platform_defaults.bash: if [[ ${HOSTNAME} == frontera ]] || [[ ${TACC_SYSTEM} == frontera ]]

 qlimits
Current queue/partition limits on TACC's Frontera system:

            Name       MinNode       MaxNode   PreemptExemptTime     MaxWall     MaxNodePU MaxJobsPU MaxSubmit 
            flex             1           128            01:00:00  2-00:00:00          6400       100       200 
     development                          40                        02:00:00            40         1         3 
          normal             3           512                      2-00:00:00          2048       100       200 
           large           513          2048                      2-00:00:00          4096         1        20 
           debug                        8368                      2-00:00:00          8368        30        60 
             rtx                          22                      2-00:00:00            22        15        45 
  rtxdevelopment                           2                        02:00:00             2         1         3 
          nvdimm                           4                      2-00:00:00             6         2         8 
           small             1             2                      2-00:00:00            30        20       200 
           grace                          30                      5-00:00:00            30        30       200 
          corral                         512                      2-00:00:00          2048       100       200 
Ovec8hkin commented 3 years ago

There's a gross way to do this without having a conditional statement. I can scan through the table headers and figure out which column is for the number of jobs (identified by the presence of 'Job' in the header) and then keep the existing logic. I can look into doing this tomorrow.

Ovec8hkin commented 3 years ago

Are their plans to use this script beyond TACC systems? If so, we cant use qlimits because that some native TACC system tool that doesn't exist on external systems.

falkamelung commented 3 years ago

We are only at TACC for now. Need a different solution for other systems.