NVIDIA / NVFlare

NVIDIA Federated Learning Application Runtime Environment
https://nvidia.github.io/NVFlare/
Apache License 2.0
648 stars 181 forks source link

Support Aux Message and Object Streaming in SP and CP #3068

Closed yanchengnv closed 4 days ago

yanchengnv commented 6 days ago

Fixes # .

Description

Object streaming is built on top of Aux Messaging. However aux messaging is currently not fully implemented for SP and CP processes.

This PR adds the support of aux messaging and object streaming to SP and CP. This is necessary when, for example, the CP needs to stream log files to the SP.

The AuxRunner currently only works for CJ and SJ: the FQCNs of the message targets are hard-coded for job processes (SJ and CJ). It is modified to compute the target FQCN based on the Process Type of the current process (SP, CP, SJ, CJ).

All engine types now support aux messaging and object streaming.

Also added no_wait version of streaming in stream_runner for future use.

Types of changes

yanchengnv commented 5 days ago

/build

YuanTingHsieh commented 5 days ago

/build