search

QMCPACK / qmcpack

Main repository for QMCPACK, an open-source production level many-body ab initio Quantum Monte Carlo code for computing the electronic structure of atoms, molecules, and solids with full performance portable GPU support
http://www.qmcpack.org
Other
297 stars 139 forks source link

Nexus: should default binding option on Summit be -b rs? #3222

Open Paul-St-Young opened 3 years ago

Paul-St-Young commented 3 years ago

Describe the bug Nexus' default bind option on Summit is -b rs. I observed that -b rs oversubscribes the cores in a pure MPI run.

To Reproduce Steps to reproduce the behavior:

  1. Launch interactive session on Summit bsub–W 10 –nnodes1 –P $OLCF_PROJECT_ID –Is $SHELL
  2. Load job-step-viewer module load job-step-viewer
  3. View jsrun with -b rs

Expected behavior All MPI tasks of a resource set are bound to its first core. Command: export OMP_NUM_THREAEDS=1; jsrun -n 6 -c 7 -a 7 -b rs -d packed brs

The problem can be solved by using -b packed:$OMP_NUM_THREADS instead of -b rs Command: export OMP_NUM_THREAEDS=1; jsrun -n 6 -c 7 -a 7 -b packed:1 -d packed bind1

Is this a real problem or did I misunderstand jsrun?

System:

Paul-St-Young commented 3 years ago

I should add that -b rs works as intended when each resource set is purely OpenMP export OMP_NUM_THREAEDS=7; jsrun -n 6 -c 7 -a 1 -b rs -d packed appropriately binds one thread to each core

ye-luo commented 3 years ago

You may use qmc-check-affinity to access exact affinity. SMT1 not used?