Unable to pass PGMPITuneLib options while running OSU Micro-Benchmarks

[chimengqiaoyu]

Dear Developer,

I am encountering some issues while using PGMPITuneLib for MPI optimization and would appreciate your guidance.

When I compile my MPI program with mpicc and specify the PGMPITuneLib option on the command line when running the mpi program, such as:

mpicc test2.c -o test100 -lpgmpitunecli -lmpi

srun -p thcp3 -N 2 -n 4 ./test100 --module=allreduce=alg:allreduce_as_reducescatter_allgatherv

the program runs and the specified algorithm is used.

#@pgmpi alg MPI_Allreduce 4 allreduce_as_reducescatter_allgatherv
#@pgmpi alg MPI_Bcast 4 default
#@pgmpi alg MPI_Reduce 8 default
#@pgmpi config size_msg_buffer_bytes 100000000
#@pgmpi config size_int_buffer_bytes 200000

However, when I try to run OSU Micro-Benchmarks with similar PGMPITuneLib options, I receive an error.

./configure CC=mpicc CXX=mpicxx LDFLAGS="-L/thfs3/home/xjtu_cx/gyh/pglib/lib" LIBS="-lpgmpicli"

srun -p thcp3 -N 2 -n 4 ./osu_allgather --module=allgather=alg:allgather_as_gather_bcast

I get the following error message:

Invalid option [-]

This is the case when there is no --module option:

srun -p thcp3 -N 2 -n 4 ./osu_allgather

# OSU MPI Allgather Latency Test v7.3
# Datatype: MPI_CHAR.
# Size       Avg Latency(us)
1                       8.45
2                       8.39
4                       8.40
8                       8.31
16                      8.37
32                      8.65
64                     10.01
128                    11.02
256                    11.62
512                    12.32
1024                   14.13
2048                   17.33
4096                   23.18
8192                   36.80
16384                  61.05
32768                  79.76
65536                 142.03
131072                280.37
262144                545.91
524288               1156.93
1048576              2282.38
#@pgmpi alg MPI_Allgather 8 default
#@pgmpi alg MPI_Allgather 1048576 default
#@pgmpi alg MPI_Allgather 32 default
#@pgmpi alg MPI_Allgather 256 default
#@pgmpi alg MPI_Allgather 32768 default
#@pgmpi alg MPI_Allgather 4 default
#@pgmpi alg MPI_Allgather 16 default
#@pgmpi alg MPI_Allgather 8192 default
#@pgmpi alg MPI_Allgather 2048 default
#@pgmpi alg MPI_Allgather 2 default
#@pgmpi alg MPI_Allgather 16384 default
#@pgmpi alg MPI_Allgather 524288 default
#@pgmpi alg MPI_Allgather 1024 default
#@pgmpi alg MPI_Allgather 131072 default
#@pgmpi alg MPI_Allgather 64 default
#@pgmpi alg MPI_Allgather 65536 default
#@pgmpi alg MPI_Allgather 4096 default
#@pgmpi alg MPI_Allgather 128 default
#@pgmpi alg MPI_Allgather 512 default
#@pgmpi alg MPI_Allgather 262144 default
#@pgmpi alg MPI_Allgather 1 default
#@pgmpi alg MPI_Reduce 8 default
#@pgmpi config size_msg_buffer_bytes 100000000
#@pgmpi config size_int_buffer_bytes 200000

Could you please advise on how to apply PGMPITuneLib optimizations when running OSU Micro-Benchmarks?I am not familiar with the detailed tuning process of PGMPITuneLib.Is there a recommended method to allow OSU benchmarks to recognize these command-line options, or should I use a different approach, such as a specific configuration file to control the tuning strategies?

Thank you very much for your assistance!

[hunsa]

Try the updated code and use env variables like

export PGMPI_PARAMS="--module=allgather=alg:allgather_as_gather_bcast"
mpirun -np 2 ./mympicode

[hunsa]

With OSU, you can try the following:

~/osu-micro-benchmarks-7.4$ PGMPI_PARAMS="--module=allreduce=alg:allreduce_as_reducescatter_allgatherv" LD_PRELOAD=../pgmpitunelib-private/lib/libpgmpicli.so srun -N2 --ntasks-per-node=2 ./c/mpi/collective/blocking/osu_allreduce -m 2

[chimengqiaoyu]

Thank you for your reply!! I will try it and keep following up