Open jafermarq opened 1 year ago
I want to try and implement FedCVAE-KD and FedCVAE-Ens and replicate the results in the paper that was provided for the MNIST and Fashion-MNIST at the varying statistical heterogeneity ($\alpha$) levels, and the varying $m$ number of clients.
Hi @conorhassan, it's great you are interested in implementing FedCVAE
using Flower. So your plan would be to reproduce experiments in Tables 2 & 3 for MNIST and Fashion-MNIST? What will you use as baseline? FedAvg?
It would be the first two third of tables 1 and 2 (i.e., not the rows corresponding to the SVHN data set). Yes, initially consider FedAvg, but ideally, a nice comparison would be to a method that is a little more tailored to a one-shot approach than FedAvg.
I know I am biased here, but the methods in this paper https://arxiv.org/abs/2302.03314 can be used for one-shot FL and maybe a more competitive comparison than FedAvg. I can hopefully contribute to this in the future.
Hi @conorhassan , sorry for the delay getting back to you. Your contribution plan makes sense, let's start with just FedAvg
? Just a quick question I'm asking to all recent contributors: are you confident you have access to the necessary compute resources (i.e. GPUs) to replicate these experiments?
It would be amazing to include your own work as a baseline here too (up to you really). Or why not having it as a standalone baseline after the Summer of Reproducibility? (your results in Table 1 look pretty good)
I'm confident that I have access to the necessary compute
Ok. Thank you for confirming. Then, FedCVAE
is yours to reproduce with Flower. I have now ✅ all points in Steps 1 & 2 above, added you as the assignee to this issue and moved it to In Progress status. You'll find how to start with your baseline by following the guide linked in the What happens next? section above. If you have any doubts (no matter how simple or complex) please reach out to me or other contributors via the Flower Slack.
Looking forward to seeing FedCVAE
in action!
FedCVAE
Do you want to work on this baseline?
What follows are the steps 1 & 2 in the Summer of Reproducibility instructions.
1. Join the Summer of Reproducibility program
#summer-of-reproducibility
.2. Define the scope of your contribution
[x] Check if you are eligible for a reward.
If where you are based is not on the list, please send us an email (
summer@flower.dev
) letting us know a bit about yourself (where are you currently based?, are you a university student? do you work at a public institution?). Please tell us the baselines you are interested in implementing (i.e. tell us your GitHub issue if you have crated one). We will reach back to you.What happens next?
[x] This item will be moved to the
In Progress
stage by a member of the Flower Team.[ ] Follow the instructions for creating a new baseline which will guide you through the process step-by-step.
Is something wrong or not clear ?