adap / flower

Flower: A Friendly Federated AI Framework
https://flower.ai
Apache License 2.0
5.15k stars 881 forks source link

docs(framework) Update how to aggregate evaluation results documentation #4529

Closed dstripelis closed 3 days ago

dstripelis commented 4 days ago
jafermarq commented 3 days ago

@dstripelis @chongshenng one thought: should we re-write the page and show how to use the evaluate_metrics_aggregation_fn callback instead? We aggregate metrics in such way in quite a few examples already: https://github.com/adap/flower/blob/main/examples/quickstart-pytorch/pytorchexample/server_app.py

I'm not the biggest fan of callbacks but writing a custom strategy for this might be overkill.

dstripelis commented 3 days ago

@jafermarq thank you for your feedback! In order to have some isolation between PRs, in the current PR I only improve the custom strategy class and address @chongshenng's comments.

However, following your suggestions, I created a new PR where I introduce the evaluation callback function in the documentation: https://github.com/adap/flower/pull/4544, while keeping the custom class strategy. I think we can review and include this PR in the next release.

chongshenng commented 3 days ago

@dstripelis @jafermarq I agree, let's focus on updating the docs for the 1.13 release, then in a subsequent PR we introduce the callback too.