FedSkip-Combatting-Statistical-Heterogeneity-with-Federated-Skip-Aggregation

This is the code for paper(ICDM22 Regular Paper) [FedSkip: Combatting Statistical Heterogeneity with Federated Skip Aggregation].

Dependencies

PyTorch >= 1.0.0
torchvision >= 0.2.1
scikit-learn >= 0.23.1

Data Preparing

Cifar-10 and Cifar100 will be automatically downloaded in your datadir while for femnist, shakespeare and synthetic, you should refer to LEAF or download our split version and unzip in the datadir/. Using LEAF to repeat our split, please refer: 1) generate a small-sized dataset of FEMNIST and full-sized datasets of SYNTHETIC and SHAKESPEARE with help of LEAF 2) remove clients with less than 64 training samples(batch size of local training).

Model Structure

For Cifar10, Cifar100, Femnist, we use the same model structure as MOON.

For SHAKESPEARE, we adopt two-layer LSTM classifier containing 100 hidden units with an 8D embedding layer according to FedProx and LEAF.

The model of SYNTHETIC is the same as LEAF: a perceptron with sigmoid activations

Parameters

Parameter	Description
`skip`	Number of skip between two aggregations.
`model`	The model architecture. Options: `simple-cnn`, `resnet50`.
`dataset`	Dataset to use. Options: `cifar10`. `cifar100`, `femnist`,`shakespeare`,`synthetic`
`lr`	Learning rate.
`batch-size`	Batch size.
`epochs`	Number of local epochs.
`n_parties`	Number of parties.
`sample_fraction`	the fraction of parties to be sampled in each round.
`comm_round`	Number of communication rounds.
`beta`	The concentration parameter of the Dirichlet distribution for non-IID partition. Setting 100000 as IID
`datadir`	The path of the dataset.
`logdir`	The path to store the logs.
`seed`	The initial seed.

For Cifar-10 and FEMNIST, you should use simple-cnn while for CIFAR-100, you should use resnet50. You can set beta as large as possible to simulate IID when partition=non-iid. We set lr=0.01, epochs=10 and batch-size=64 by default in the paper.

Usage

Here is an example to run FedSkip-3 on CIFAR-10 with a simple CNN:

python main.py --dataset=cifar10 \
    --skip=3 \
    --lr=0.01 \
    --epochs=10 \
    --model=simple-cnn \
    --comm_round=100 \
    --n_parties=10 \
    --beta=0.5 \
    --sample_fraction=1.0 \
    --logdir='./logs/' \
    --datadir='./data/' \

Acknowledgement

We borrow some codes from MOON, LEAF and FedProx

Attention

Here we provide code of FedSkip for cifar10 and cifar100. Other datasets and methods will be updated soon.

Contact

If you have any problem with this code, please feel free to contact zqfan_knight@sjtu.edu.cn.

MediaBrain-SJTU / FedSkip

readme