Clarification on comparison of local models vs federated models?

[darpitdavetamu]

Where you compare the performance of the local models vs the federated model. The weights for the local models are based on training on only local data or it involves the weights received by it from the global model?

[phil8192]

between each federated averaging round, the weights for each of the local models are initialised with the global weights.

[darpitdavetamu]

@phil8192 Thank you for your response. That helps a lot. Also, if the number of rounds is kept to 1. The weights for the local model at the end of round 1 would be global weights or derived only from local models?

[phil8192]

they'd be derived only from the local models :+1:

[darpitdavetamu]

@phil8192 I took a while to thoroughly inspect things. I trained a federated model with your code and kept number of rounds to 1. On other hand, I do the same process in a non-federated structure. That is, I train local models for each runner and average the weights to get a comparable global model.

When I compare the global models (federated vs non-federated) there is negligible difference. However, when I compare local models individually, there is a huge difference. Please find the below attached statistics for local models trained from your code vs training them individually. Can you please help me understand why this might arise.

"Fed Local ..." - local models with your code "Central Fed Local ..." - individually trained models