search

Bluefog-Lib / bluefog

Distributed and decentralized training framework for PyTorch over graph
https://bluefog-lib.github.io/bluefog/
Apache License 2.0
291 stars 71 forks source link

Enhancement Thoughts #1

Closed BichengYing closed 4 years ago

BichengYing commented 4 years ago
  1. Add push-sum implementation to avoid bias?
  2. Time-varying topology? Communicated with only one or two neighbors in each iteration
BichengYing commented 4 years ago
  1. Support quantized data type, such as half-float, uint8, uint16, etc.
BichengYing commented 4 years ago
  1. Add the weights for neighbor averaging
BichengYing commented 4 years ago

Add the weights for neighbor averaging is done.

BichengYing commented 4 years ago
  1. Push-sum algorithm is done. Win_accumulate and Win_Mutex are introduced to achieve this
BichengYing commented 4 years ago

2 & 4 are done. New API is introduced in win_put/get/accumulate. A destination/source_weights dictionary can be used to call the API.

BichengYing commented 4 years ago

Almost all fulfilled. The support for quantized data type will be considered in the future