Question about parameter fl_eta in cifar_fed.yaml

[LukeZheZhu]

Hi @ebagdasa,

Thanks for sharing code.

I am trying to run cifar_fed with command,

    python training.py --name cifar --params configs/cifar_fed.yaml --commit none

I am a little confused about the parameter fl_eta.

In function, run_fl_round (training.py) , the variable, round_participants,

    round_participants = hlpr.task.sample_users_for_round(epoch)

uses parameter fl_no_models (cifar_fed.yaml) to decide the number of users updating weights to server, for example 10 in cifar_fed.yaml.

Then, the code

    hlpr.task.update_global_model(weight_accumulator, global_model)

calls the function update_global_model (fl_task.py).

In function update_global_model (fl_tas.py),

    def update_global_model(self, weight_accumulator, global_model: Module):
        for name, sum_update in weight_accumulator.items():
            if self.check_ignored_weights(name):
                continue
            scale = self.params.fl_eta / self.params.fl_total_participants
            average_update = scale * sum_update
            self.dp_add_noise(average_update)
            model_weight = global_model.state_dict()[name]
            model_weight.add_(average_update)

the sum_update is the sum of all users' weights, which is supposed to be divided by the value of fl_no_model. In the code, however, you use variables scale

    scale = self.params.fl_eta / self.params.fl_total_participants
    average_update = scale * sum_update

to process the sum_update. I didn't find any explains of this logic in papers or any comments in the code.

I wonder would you mind giving more details about the usage of fl_eta? My questions are,

1. Why sum_update doesn't divide fl_no_model?
2. What is the meaning of self.params.fl_eta / self.params.fl_total_participants?
3. How should I set the fl_eta, if I trying to increase the value of fl_no_model?

Thanks

[ebagdasa]

yep, will check this week.

[LukeZheZhu]

yep, will check this week.

Thanks. Really appreciate.