TL-System / plato

A federated learning framework to support scalable and reproducible research
Apache License 2.0
343 stars 80 forks source link

[RFC] Found no advantages in model fusion methods #126

Closed xujli closed 2 years ago

xujli commented 2 years ago

I use this framework to reproduce some algorithms which are helpful to accelerate convergence in iid and non-iid data distribution setting, such as FedAdp, Fedatt, FedProx, all these algorithms are in your ./examples dir. I ran these algorithms and plot the accuracy on test dataset, and found that the curves have almost no difference. There is no bug or execution confuse when I check the code. So I wonder if you can show me some examples and settings that these algorithms perform a more significant advance on convergence than FedAvg. It would be a great pleasure for me if you give me some brief explanations and advise to reproduce these algorithm.

Best Regards.

baochunli commented 2 years ago

This is related to the performance of these algorithms that are designed by the authors of their relevant papers, rather than issues of implementations in Plato. The implementations in examples/ are contributed by various contributors, and serve as references and examples for using Plato as a framework. One possible idea towards showing different results is to use more sophisticated models (rather than MNIST), and to introduce non-iid data distributions (using samplers provided by Plato) with the concentration parameter α defined appropriately in the corresponding Dirichlet distribution.

xujli commented 2 years ago

Thanks for your reply, I'll try more settings and models. I just want to get some advise on my results reproduction and I'm not familiar with the rules of Issues. I apologize for my wrong use. If I reproduce some other algorithms correctly, may I commit PR to the Repo?

baochunli commented 2 years ago

Of course.