search

hpcg-benchmark / hpcg

Official HPCG benchmark source code
http://www.hpcg-benchmark.org/
BSD 3-Clause "New" or "Revised" License
293 stars 123 forks source link

Add ability to execute hpcg with different but geometrically compatible local grid sizes #46

Open maherou opened 8 years ago

maherou commented 8 years ago

Heterogeneous machines will need to have different local grid sizes to help balance load. We can support varying grid sizes as long as dimensions are compatible across local grid faces.

maherou commented 7 years ago

The first version of this feature will work for a 2-block partition of the z processor dimension. It works by specifying three new arguments to the xhpcg command:

--pz - the starting z-dimension process ID for the second block. --zl - the value of nz in the lower block of the z processor dimension --zu - the value of nz in the upper block

where nz is the local grid dimension in the z-dimension.

Example: If running on 1000 MPI processes, the processor grid would be 10 by 10 by 10. If you want to have different sizes for nz so that you could balance the load on a heterogeneous machine, such that nz is 64 in the lower block and 128 in the upper block, use

--pz=5 --zl=64 --zu=128

You will also need to launch the MPI program in MPMD mode passing in the compatible values for nz in both the lower and upper block. This is perhaps some tedious bookkeeping but should work.