search

litian96 / fair_flearn

Fair Resource Allocation in Federated Learning (ICLR '20)
MIT License
242 stars 59 forks source link

qffedavg.py doesn't match with Alg 2. #13

Closed omarfoq closed 3 years ago

omarfoq commented 3 years ago

Hello,

I am not sure about this, but it seems that Alg. 2 (In the paper) does not match exactly the implementation in [flearn/trainers/qffedavg.py](https://github.com/litian96/fair_flearn/blob/master/flearn/trainers/qffedavg.py). Specifically, in Line 62 you have

hs.append(self.q * np.float_power(loss+1e-10, (self.q-1)) * norm_grad(grads) + (1.0/self.learning_rate) * np.float_power(loss+1e-10, self.q))

While Alg. 2 suggests that you should have

hs.append(self.q * np.float_power(loss+1e-10, (self.q-1)) * np.power(norm_grad(grads), 2) + (1.0/self.learning_rate) * np.float_power(loss+1e-10, self.q))

litian96 commented 3 years ago

Hello,

Thanks for asking. norm_grad is actually the square of the L2 norm (the naming is a bit confusing...) (see here)

Please let me know if you have other questions.