EF21

[jafermarq]

EF21

• Title: EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback
• Venue: NeurIPS 2021
• Link to paper: https://proceedings.neurips.cc/paper/2021/hash/231141b34c82aa95e48810a9d1b33a79-Abstract.html

Do you want to work on this baseline?

🌻 Check everything about the Summer of Reproducibility on flower.dev/summer

All available baselines are listed in the Summer of Reproducibility Dashboard and also in the GitHub Issues with the summer-of-reproducibility label. The content is the same.

📝 It is advised to complete these steps before your start working on your code. But if you can't wait to implement your baseline with Flower (we totally understand it 😄), please ensure you follow the steps on how to contribute a new baseline.

What follows are the steps 1 & 2 in the Summer of Reproducibility instructions.

1. Join the Summer of Reproducibility program

• [x] Join the Flower Slack and say "hi! 👋" in the channel #summer-of-reproducibility.
• [x] Pick a baseline from our curated list <---------------------------------------- [you are doing this now]

  2. Define the scope of your contribution

• [x] What are you going to reproduce? Add a comment to your issue and tell us about your plan regarding this baseline: what experiments from the paper are you reproducing?, for which datasets ? the more details you provide us with the better !

• [x] Check if you are eligible for a reward.

  As we have to comply with US/EU regulations, we have checked that individual contributors based on these countries or territories are eligible: Australia, Austria, Belgium, Bulgaria, Canada, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Gibraltar, Greece, Hong Kong SAR China, Hungary, India, Ireland, Italy, Japan, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, Mexico, Netherlands, New Zealand, Norway, Poland, Portugal, Romania, Singapore, Slovakia, Slovenia, South Korea, Spain, Sweden, Switzerland, Thailand, United Arab emirates, United Kingdom United States.

  If where you are based is not on the list, please send us an email (summer@flower.dev) letting us know a bit about yourself (where are you currently based?, are you a university student? do you work at a public institution?). Please tell us the baselines you are interested in implementing (i.e. tell us your GitHub issue if you have crated one). We will reach back to you.

• [x] We will discuss with you about your contribution plan, if it sounds like a substantial enough contribution according to the Summer of Reproducibility rules (check our website flower.dev/summer), you'll get the OK from us to start working on your baseline!

What happens next?

• [x] This item will be moved to the In Progress stage by a member of the Flower Team.

• [ ] Follow the instructions for creating a new baseline which will guide you through the process step-by-step.

Is something wrong or not clear ?

• Ask a question directly in your issue.
• Reach out to us via the Flower Slack and ask your question in the #summer-of-reproducibility channel
• Check all the details (including FAQ) in the Summer of Reproducibility website: flower.dev/summer

[burlachenkok]

Hi! My name is Konstantin Burlachenko (Link to my GitHub account: https://github.com/burlachenkok)

I'm planning to work on this project.

What are you going to reproduce?

I'm planning to reproduce the experiments from the paper.

Specifically, I plan to reproduce:

• All experiments from "Section 5. Experiments" of the main part of the paper. (Convex logistic regression)
• Experiments with nonconvex logistic regression from section A.1 (nonconvex logistic regression)

The authors analyzed step size tolerance by plotting how the norm of gradient square changes with the number of sent bits per client via increasing step size by factors x1,x8,x16,x32,x128.

My goal will be to reproduce experiments considering all three algorithms mentioned in the paper for experiment:

• classical EF
• EF21
• EF21+ (EF21+ is extension of EF21)

[jafermarq]

Hi @burlachenkok , your contribution plan looks good. Including the experiments shown in Section 5 (i.e. Figure 1 & 2) make sense. How about including the deep learning experiments from Figures 13 and/or 14 in the Appendix before including those in A.1? Showing EF21 under experimental setups that are more familiar to others working in FL (typically with CIFAR, FEMNIST, etc dataset) would be very valuable for the community! Let me know what you think about this change.

[burlachenkok]

How about including the deep learning experiments from Figures 13 and/or 14 in the Appendix

@jafermarq Sure, let's include this. Let's replicate these figures for:

• classical EF
• EF21
• EF21+

I'd like to exclude SGD, in the paper, because:

• It's unclear which variant of SGD has been used exactly.
• The SGD is not the main goal of the paper.

Please let me know if I can implement three Algorithms (EF21, EF21+, Classical EF) for Fig. 13, and 14 from the Paper, and it is good enough?

Thanks.

[jafermarq]

@burlachenkok , yeah. excluding the SGD from Fig 13 and 14 is fine. But are you still considering including Fig 1 & 2 also?

[burlachenkok]

@jafermarq, thank you. Yes, it includes Fig.1, Fig.2. So finally I plan to reproduce experiments that are demonstrated in Fig.1, Fig.2, Fig.13, and Fig.14.

[jafermarq]

@burlachenkok perfect! I have now ✅ all the points in Step 1 & 2 and added you as the assignee to this issue. Following the link in What happens next? section you'll find the details on how to start with the code for your baseline. Please reach out over Slack if you have some questions about the process or about Flower.

Looking forward to see EF21 in action !