search

KarhouTam / FL-bench

Benchmark of federated learning. Dedicated to the community. 🤗
GNU General Public License v3.0
505 stars 82 forks source link

Evaluation in test phase #21

Closed butcher1226 closed 1 year ago

butcher1226 commented 1 year ago

In the test phase, it seems that there is only average of local test on clients, but no global test on the server?

KarhouTam commented 1 year ago

Hi, @butcher1226. First of all, thanks for your attention to FL-bench.

For your question, first, the setting of the global testset held by the server is incompatible with pFL methods, whose goal is for each client to minimize the empirical risk to their local testset; second, for traditional FL methods, the weighted average results of the client's local testset accuracy is equal to the accuracy of a single global testset. More explanation is given in #14 .

KarhouTam commented 1 year ago

This issue is closed due to long time no response.