IBM Federated Learning
What is it?
IBM federated learning is a Python framework for federated learning (FL) in an enterprise environment. FL is a distributed machine learning process, in which each participant node (or party) retains data locally and interacts with the other participants via a learning protocol. The main drivers behind FL are privacy and confidentiality concerns, regulatory compliance requirements, as well as the practicality of moving data to one central learning location.
IBM federated learning provides a basic fabric for FL, to which advanced features can be added. It is not dependent on any specific machine learning framework and supports different learning topologies, e.g., a shared aggregator, and protocols. It supports Deep Neural Networks (DNNs) as well as classic machine learning techniques such as linear regression and k-means. This comprises supervised and unsupervised approaches as well as reinforcement learning. The figure below shows a typical configuration of an aggregator based federated learning setup supported by IBM federated learning.
A set of parties own data and each trains a local model. The parties exchange updates with an aggregator using a FL protocol. The aggregator fuses (aggregates) the results from the different parties and ships the consolidated results back to the parties. This can go through multiple rounds until a termination criterion is reached. IBM federated learning supports the configuration of these training scenarios.
The key design points of IBM federated learning are the ease of use for the machine learning professional, configurability to different computational environments - from data centers to edge devices - and extensibility. It can be extended to work with different machine learning (ML) libraries, learning protocols, and fusion algorithms. This provides a basic fabric on which FL projects can be run and research in FL learning can take place.
IBM federated learning comes with a large library of fusion algorithms for both DNNs and classic ML approaches, consisting of implementations of both common, published fusion algorithms as well as novel ones we have developed.
Supported functionality
IBM federated learning supports the following machine learning model types:
- Neural networks (any neural network topology supported by Keras, PyTorch and Tensorflow)
- Decision Tree ID3
- Linear classifiers/regressions (with regularizer): logistic regression, linear SVM, ridge regression, Kmeans and Naïve Bayes
- Deep Reinforcement Learning algorithms including DQN, DDPG, PPO and more
IBM federated learning supports multiple state-of-the-art fusion algorithms to combine model updates coming from multiple parties. Changes in this algorithm may speed up the convergence, reduce training time or improve model robustness. For a particular ML model, you can select multiple types of fusion algorithms:
| Supported ML Models | Supported fusion algorithms |
|---|---|
| Neural Networks | Iterative Average |
| FedAvg McMahan et al. | |
| Gradient Average | |
| PFNM Yurochkin et al. | |
| Krum Blanchard et al. | |
| Coordinate-wise median Yin et al. | |
| Zeno Xie et al. | |
| SPAHM Yurochkin et al. | |
| Fed+ Yu et al. | |
| FedProx: Tian Li et al. | |
| Shuffle Iterative Average Cheng et al. | |
| Comparative Elimination Gupta et al. | |
| Adaptive Federated Averaging Muñoz-González et al. | |
| ID3 Decision Tree | ID3 fusion Quinlan |
| Reinforcement Learning RLLib models | Iterative Average |
| FedAvg McMahan et al. | |
| Linear classifiers with SGD | Iterative Average |
| K-means | SPAHM Yurochkin et al. |
| Naïve Bayes | Naive Bayes fusion with differential privacy |
We also support the following fairness techniques that help to mitigate bias in federated learning and can be coupled for multiple types of ML models:
| Fairness techniques | Algorithm types | Supported ML models |
|---|---|---|
| Local Reweighing Abay et al. | Pre-processing | All ML models |
| Global Reweighing with Differetial Privacy Abay et al. | Pre-processing | All ML models |
| Federated Prejudice Removal Abay et al. | In-processing | Logistic Regression |
In order to aid orchestration of Federated Learning experiments using the IBMFL library, we also provide a Jupyter Notebook based UI interface, Experiment Manager Dashboard where users can choose the model, fusion algorithm, number of parties and other (hyper) parameters for a run. This orchestration can be done on the machine where the notebook is hosted, i.e., locally or even across remote machines. The usage guide on how to go about using the dashboard can be found here.
IBMFL Multi-Cloud and Hybrid Cloud Orchestrator automates the deployment and monitoring of aggregator and party process using federated learning library docker image on OpenShift clusters which are setup on different cloud data center regions. For more information on how to use OpenShift Orchestrator please refer to README.
How to get started?
Clone the repository. The main framework runtime is packaged in a whl file.
Try the set-up guide for a single-node federated learning setup.
Try the crypto set-up guide for running federated learning with fully homomorphic encryption (HE).
There are a number of examples with explanation for different federated learning tasks with different model types to get started with.
Playing with IBM FL via pre-configured examples on popular datasets, e.g, MNIST, CIFAR10, FEMNIST and Adult, etc.
Try our experiment manager here.
Try IBM FL with OpenShift here.
How does it work?
There is a docs folder with tutorials and API documentation to learn how to use and extend IBM federated learning. We also have a few video tutorials.
- Web Site
- Aggregator and party configuration tutorial
- API documentation
- Related publications
- White paper
How to get in touch?
We appreciate feedback and questions. Please post issues when you encounter them.
We have set up a Slack channel for ongoing discussion. Join the IBM federated learning workspace: https://ibm-fl.slack.com/. If the previous link does not work for you, you can also use this invitation link.
Citing IBM Federated Learning
If you use IBM Federated Learning, please cite the following reference paper:
@article{ibmfl2020ibm,
title={IBM Federated Learning: an Enterprise Framework White Paper V0. 1},
author={Ludwig, Heiko and Baracaldo, Nathalie and Thomas, Gegi and Zhou, Yi and Anwar, Ali and Rajamoni, Shashank and Ong, Yuya and Radhakrishnan, Jayaram and Verma, Ashish and Sinn, Mathieu and others},
journal={arXiv preprint arXiv:2007.10987},
year={2020}
}Ongoing effort
This is an ongoing effort. We plan to update this repo as new functionality is added frequently.
License
The distribution of IBM federated learning in this repository is for non-commercial and experimental use under this license. For commercial use IBM federated learning is available in IBM CloudPak for Data and as a service.