FedSAM

[jafermarq]

FedSAM

• Title: Improving Generalization in Federated Learning by Seeking Flat Minima
• Venue: ECCV 2022
• Link to paper: https://arxiv.org/abs/2203.11834

Do you want to work on this baseline?

🌻 Check everything about the Summer of Reproducibility on flower.dev/summer

All available baselines are listed in the Summer of Reproducibility Dashboard and also in the GitHub Issues with the summer-of-reproducibility label. The content is the same.

📝 It is advised to complete these steps before your start working on your code. But if you can't wait to implement your baseline with Flower (we totally understand it 😄), please ensure you follow the steps on how to contribute a new baseline.

What follows are the steps 1 & 2 in the Summer of Reproducibility instructions.

1. Join the Summer of Reproducibility program

• [x] Join the Flower Slack and say "hi! 👋" in the channel #summer-of-reproducibility.
• [x] Pick a baseline from our curated list <---------------------------------------- [you are doing this now]

  2. Define the scope of your contribution

• [x] What are you going to reproduce? Add a comment to your issue and tell us about your plan regarding this baseline: what experiments from the paper are you reproducing?, for which datasets ? the more details you provide us with the better !

• [x] Check if you are eligible for a reward.

  As we have to comply with US/EU regulations, we have checked that individual contributors based on these countries or territories are eligible: Australia, Austria, Belgium, Bulgaria, Canada, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Gibraltar, Greece, Hong Kong SAR China, Hungary, India, Ireland, Italy, Japan, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Romania, Singapore, Slovakia, Slovenia, South Korea, Spain, Sweden, Switzerland, Thailand, United Arab emirates, United Kingdom United States.

  If where you are based is not on the list, please send us an email (summer@flower.dev) letting us know a bit about yourself (where are you currently based?, are you a university student? do you work at a public institution?). Please tell us the baselines you are interested in implementing (i.e. tell us your GitHub issue if you have crated one). We will reach back to you.

• [x] We will discuss with you about your contribution plan, if it sounds like a substantial enough contribution according to the Summer of Reproducibility rules (check our website flower.dev/summer), you'll get the OK from us to start working on your baseline!

What happens next?

• [x] This item will be moved to the In Progress stage by a member of the Flower Team.

• [ ] Follow the instructions for creating a new baseline which will guide you through the process step-by-step.

Is something wrong or not clear ?

• Ask a question directly in your issue.
• Reach out to us via the Flower Slack and ask your question in the #summer-of-reproducibility channel
• Check all the details (including FAQ) in the Summer of Reproducibility website: flower.dev/summer

[debcaldarola]

Hi @jafermarq!

This is the contribution plan I'd like to propose:

• Implementation of FedSAM, FedASAM on the client-side and SWA on the server-side.
• Experiments with FedSAM and FedASAM on the federated CIFAR10 and CIFAR100 with different degrees of heterogeneity (alpha=0, 0.05/0.5, 100/1000) and clients rate of participation (5%, 10%, 20%), i.e. Table 2 in the paper.
• Experiments with SWA on the server-side under the same conditions of the previous point (Table 2).
• The baselines will be compared with FedProx, FedAvgM and their combination with SAM, ASAM and SWA (Table 6). Specific focus will be given to FedAvgM, since the server-side momentum combined with FedASAM+SWA allows the model to reach convergence even in the most heterogeneous scenarios. Potentially, SCAFFOLD, FedDyn and Adabest may be added as well (lower priority).

Looking forward to contribute! Happy to discuss this further!

[jafermarq]

Hi @debcaldarola, what you propose to implement sounds perfect! I agree, leaving SCAFFOLD, FedDyn and Adabest` as lower priority makes sense.

Just one question: could you please just confirm that you are ok with what's state in the reward eligibility point (second point in Step 2)?

[debcaldarola]

Perfect! :)

I confirm you that I am eligible for rewards!

[jafermarq]

Amazing! thanks for confirming. I've now ✅ all the items in Step 1 &2. The link in what happens next? will take you to a guideline showing how to start the process of creating your FedSAM baseline. Feel free to reach out to us either adding a comment here or on Slack.

Looking forward to seeing FedSAM in action ! 🙌

[debcaldarola]

Awesome! Thanks :D