Add Flower Baseline: Ditto

oscardilley commented 1 month ago

Paper

"Ditto: Fair and Robust Federated Learning Through Personalization" by Tian Li, Shengyuan Hu, Ahmad Beirami, Virginia Smith

Link

https://arxiv.org/abs/2012.04221

Maybe give motivations about why the paper should be implemented as a baseline.

From what I can see it was proposed for the summer of reproducibility but has not yet been contributed:

https://github.com/adap/flower/issues/2026#issue-1791796534

I have implemented this as a part of other research and would like to contribute the work.

Is there something else you want to add?

No response

Implementation

To implement this baseline, it is recommended to do the following items in that order:

For first time contributors

[x] Read the first contribution doc
[x] Complete the Flower tutorial
[x] Read the Flower Baselines docs to get an overview:
- [x] How to use Flower Baselines
- [x] How to contribute a Flower Baseline

Prepare - understand the scope

[x] Read the paper linked above
[ ] Decide which experiments you'd like to reproduce. The more the better!
[ ] Follow the steps outlined in Add a new Flower Baseline.
[ ] You can use as reference other baselines that the community merged following those steps.

Verify your implementation

[ ] Follow the steps indicated in the EXTENDED_README.md that was created in your baseline directory
[ ] Ensure your code reproduces the results for the experiments you chose
[ ] Ensure your README.md is ready to be run by someone that is no familiar with your code. Are all step-by-step instructions clear?
[ ] Ensure running the formatting and typing tests for your baseline runs without errors.
[ ] Clone your repo on a new directory, follow the guide on your own README.md and verify everything runs.

jafermarq commented 1 month ago

Hi @oscardilley your are right, Ditto was one of the baselines proposed for Summer of Reproducibility but it wasn't completed. Would you like to add it to Flower Baselines? Do you need any help?

oscardilley commented 1 month ago

Hey @jafermarq, yes, I would really like to contribute it as a baseline - I have started reconfiguring it into the required format. I think I am getting on well so far, but is this the best channel to ask any future questions I may have?

oscardilley commented 1 month ago

Hi @jafermarq ,

The reproduction of Ditto is going well, but I have a couple of sticking points that I was hoping you could advise on...

From the paper https://arxiv.org/abs/2012.04221 + repo https://github.com/litian96/ditto, they don't seem to provide some of hyperparameters used to achieve the reported results, specifically:

Whether the stated learning rate applies to the local and global objectives, in relation to Algorithm 2.
The number of clients selected for each round of training.
The number of local iterations to update the global and personalised parameters.

With this in mind, what level of result reproduction do you expect?

Further, would it be acceptable, for the contribution, to only replicate a subset of the results. i.e. the 'clean' tests without clients acting maliciously?

Thanks!

On Fri, Jul 12, 2024 at 10:05 AM Javier @.***> wrote:

Hi @oscardilley https://github.com/oscardilley your are right, Ditto was one of the baselines proposed for Summer of Reproducibility but it wasn't completed. Would you like to add it to Flower Baselines? Do you need any help?

— Reply to this email directly, view it on GitHub https://github.com/adap/flower/issues/3756#issuecomment-2225147426, or unsubscribe https://github.com/notifications/unsubscribe-auth/AO72UZXJDXILZSTV4B6JYSLZL6L6PAVCNFSM6AAAAABKS7L6X6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRVGE2DONBSGY . You are receiving this because you were mentioned.Message ID: @.***>

oscardilley commented 3 weeks ago

Hi, @jafermarq, please could you advise on this?

adap / flower