search

IBM / federated-learning-lib

A library for federated learning (a distributed machine learning process) in an enterprise environment.
Other
493 stars 134 forks source link

Multiple clients on single machine #51

Open TomKolo opened 3 years ago

TomKolo commented 3 years ago

Hi,

I was wondering if I can use IBM Federated Learning to perform some experiments on full datasets (e.g. FEMNIST from LEAF) when the number of available machines is substantially smaller than the number of client datasets. Is it possible to simulate multiple clients (>50) on one machine in other way than by separately running and registering party instance for each of them?

Thank you!

chalianwar commented 3 years ago

Hi,

Thanks for checking out our FL library!

Yes, it is definitely possible. You can launch 50 parties on a single machine as long as your machine can support that much computation. We have tested MNIST demo in the examples with 50 parties on a single machine. However, keep in mind that in some cases the machine can run out of memory or become slow if it's not a powerful one.

Please let us know if you face any problem.

HenryHu-H commented 2 years ago

Hi, I tried using threading.Thread, ibmfl.aggregator.aggregator.Aggregator and ibmfl.party.party.Party to simulate multiple terminals that are used in quickstart.md. Although the program could be run without errors, it produced results that are much worse than those produced by separate terminals (accuracy dropping by around 20%). Would you please share your code that tested MNIST demo with 50 parties on a single machine so I can use it as a reference to fix the problem with my code? Thanks a lot!