search

openucx / ucc

Unified Collective Communication Library
https://openucx.github.io/ucc/
BSD 3-Clause "New" or "Revised" License
207 stars 97 forks source link

CL/HIER allgatherv #1016

Open x41lakazam opened 2 months ago

x41lakazam commented 2 months ago

allgatherv algorithm split into three parts:

Still under development. Currently supports only int32, I will add support for int64 once I'm sure of the logic.

Situation where the ranks are not in a continuous order across nodes is not handled yet.

x41lakazam commented 2 months ago

Mpi test results:

$ mpirun         \
    -x UCC_CL_HIER_ALLGATHERV_TUNE=allgatherv:@node_split:inf \
    -x UCC_CLS=basic,hier \
    -x UCC_CL_HIER_NODE_LEADERS_SBGP_TLS=ucp \
    -x UCC_CL_HIER_NODE_SBGP_TLS=ucp \
    -x LD_PRELOAD=./lib/libucc.so \
    -x PATH         \
    -x LD_LIBRARY_PATH         \
    -np 6 --map-by ppr:3:node  \
    ./bin/ucc_test_mpi -c allgatherv -t world -d uint32 -v -m 8 -v 

===== UCC MPI TEST INFO =======
seed:         6222
collectives:  Allgatherv
data types:   uint32
memory types: Host
teams:        world
tc=Allgatherv team=world mtype=host msgsize=8 persistent=0 inplace=0 dt=uint32

===== UCC MPI TEST REPORT =====
collective                 tests    passed    failed   skipped
Allgatherv                     1         1         0         0

===== UCC MPI TEST SUMMARY =====
total tests:  1
passed:       1
skipped:      0
failed:       0
elapsed:      9s
janjust commented 2 months ago

@x41lakazam is this still draft?