search

anonymifish / fed-distribution-matching

Implementation of FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
11 stars 0 forks source link

Performance reproduction: can this code reach the test acc(e.g. fig2(b) ) claimed in the paper? #1

Open Youth-49 opened 10 months ago

Youth-49 commented 10 months ago

I wonder is it the official code implementation of the paper?

can this code achieve the performance claimed in the paper? The code set the same hyper-parameters as the paper and I'd like to know if you can reproduce the result in figure 2 in the paper.

I run the run.sh and got the result of 61% test accuracy in cifar10 under Dir_0.5(10), but FedAvg can easily achieve 70% test accuracy with local epoch=10, lr=0.01, momentum=0.5. The test accuracy curve against the communication of FedDM and FedAvg round are as below.

net vanilla_fed_cifar10_ConvNet_20_C1_iidFalse_E10_nclient10_a0 5

Ariatao commented 5 months ago

Hello, I also encountered the same problem when reproducing this repository code, but I did not achieve such a fast convergence speed using FedAvg under Dir (0.5) partitioning (about 63% test accuracy at 20th round, CIFAR10, simple-cnn). I use implementations from https://github.com/Xtra-Computing/NIID-Bench, all parameters keep default, which same as the baselines implemented in the paper FedDM. I am currently considering whether the decrease in accuracy was caused by reasons such as not performing DSA operations during the training process. If there is any progress, we can discuss it together.

aikedaerC commented 2 months ago

I got results similar to FedDM using centralized training.