Implementations of various Federated Learning (FL) algorithms in PyTorch, especially for research purposes.
torchvision.datasets
.torchtext.datasets
.TinyImageNet
, CINIC10
).BeerReviews
).Heart
, Adult
, Cover
).GLEAM
)IID
(i.e., statistical homogeneity)Unbalanced
(i.e., sample counts heterogeneity)Pathological Non-IID
(McMahan et al., 2016)Dirichlet distribution-based Non-IID
(Hsu et al., 2019)Pre-defined
(for datasets having natural semantic separation, including LEAF
benchmark (Caldas et al., 2018))
LogReg
(logistic regression), StackedTransformer
(TransformerEncoder-based classifier)TwoNN
, TwoCNN
, SimpleCNN
(McMahan et al., 2016)FEMNISTCNN
, Sent140LSTM
(Caldas et al., 2018))LeNet
(LeCun et al., 1998), MobileNet
(Howard et al., 2019), SqueezeNet
(Iandola et al., 2016), VGG
(Simonyan et al., 2014), ResNet
(He et al., 2015)MobileNeXt
(Daquan et al., 2020), SqueezeNeXt
(Gholami et al., 2016), MobileViT
(Mehta et al., 2021)DistilBERT
(Sanh et al., 2019), SqueezeBERT
(Iandola et al., 2020), MobileBERT
(Sun et al., 2020)M5
(Dai et al., 2016)
FedAvg
and FedSGD
(McMahan et al., 2016) Communication-Efficient Learning of Deep Networks from Decentralized DataFedAvgM
(Hsu et al., 2019) Measuring the Effects of Non-Identical Data Distribution for Federated Visual ClassificationFedProx
(Li et al., 2018) Federated Optimization in Heterogeneous NetworksFedOpt
(FedAdam
, FedYogi
, FedAdaGrad
) (Reddi et al., 2020) Adaptive Federated Optimizationlocal
: evaluate FL algorithm using holdout sets of (some/all) clients NOT participating in the current round. (i.e., evaluation of personalized federated learning setting)global
: evaluate FL algorithm using global holdout set located at the server. (ONLY available if the raw dataset supports pre-defined validation/test set).both
: evaluate FL algorithm using both local
and global
schemes.
requirements.txt
. (I recommend building an independent environment for this project, using e.g., Docker
or conda
)torchtext
, please check the version compatibility with torch
. (See official repository)torch
-related packages using one command provided by the official guide (See official installation guide); e.g., conda install pytorch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 torchtext==0.13.0 cudatoolkit=11.6 -c pytorch -c conda-forge
python3 main.py -h
.commands
directory.EdgeNeXt
)MedMNIST
)SuPerFed
).Should you have any feedback, please create a thread in issue tab. Thank you :)