search

Bluefog-Lib / bluefog

Distributed and decentralized training framework for PyTorch over graph
https://bluefog-lib.github.io/bluefog/
Apache License 2.0
291 stars 71 forks source link

Error when calling push-sum optimizer #112

Open yangxuanfei opened 2 years ago

yangxuanfei commented 2 years ago

Note that the bottom layer has the definition of DistributedPushSumOptimizer, why call this error： AttributeErron: module "bluefog.torch’’has no attribute 'DistributedPushSumOntimizer‘

BichengYing commented 2 years ago

Hi yangxuanfei,

It is might because of the typo? DistributedPushSumOntimizer, note you wrote Ontimizer instead of Optimizer?

The definition of this in here: https://github.com/Bluefog-Lib/bluefog/blob/master/bluefog/torch/optimizers.py#L1180

If it is note the typo, could check the versions?

Thanks

yangxuanfei commented 2 years ago

Thanks for the reply, but I checked and it wasn't a typo. Are you referring to the version problem of bluefog? We are using version 0.3.0, is this a problem?

BichengYing commented 2 years ago

Oh, we didn't expose this optimizer. See the source code here https://github.com/Bluefog-Lib/bluefog/blob/master/bluefog/torch/__init__.py#L21 Because we are not satisfied the current implementation and the performance doesn't seem very good.

However, if you really want to use it, you can try this

from bluefog.torch.optimizers import DistributedPushSumOptimizer

I will encourage you to improve the original source code if you have any idea. Code is here https://github.com/Bluefog-Lib/bluefog/blob/8f74265c50c31e3e62aa21bed74fdc98c109542e/bluefog/torch/optimizers.py#L1026