pfedme Optimizer Probelem

[adam4096]

Hello, I want to ask:
Why is different, code and _algorithm? That is in fedoptimizer.py line 64: p.data = p.data - group['lr'] (p.grad.data + group['lamda'] (p.data - localweight.data) + group['mu']*p.data) and Algorithm 1 line 8

[CharlieDinh]

Hi.

The line 64 in optimizer is to solve h_i (Equation 7 in our paper) to find the persionalized model.

Algorithm 1 line 8 is in userpFedMe.py line 53-54 to find local model.

for new_param, localweight in zip(self.persionalized_model_bar, self.local_model): localweight.data = localweight.data - self.lamda self.learning_rate (localweight.data - new_param.data)

[adam4096]

Thank you!

[adam4096]

Hi.

The line 64 in optimizer is to solve h_i (Equation 7 in our paper) to find the persionalized model.

Algorithm 1 line 8 is in userpFedMe.py line 53-54 to find local model.

for new_param, localweight in zip(self.persionalized_model_bar, self.local_model): localweight.data = localweight.data - self.lamda self.learning_rate (localweight.data - new_param.data)

Hello, For the line 64 in optimizer is to solve h_i, do you modify the NAG to find the personalized model? It doesn't look like a standard NAG update formula.

[CharlieDinh]

No. I just applied gradient descent for h_i

[adam4096]

No. I just applied gradient descent for h_i

But this update looks strange. Can you explain it? Thamk you very much! p.data = p.data - group['lr'] (p.grad.data + group['lamda'] (p.data - localweight.data) + group['mu']*p.data)

[CharlieDinh]

(p.grad.data ) is gradient update for of f_bar (group['lamda'] (p.data - localweight.data) is gradient for moreau envelopes regularization term group['mu']p.data is just norm 2 regularizartion, but mu = 0 mean we don;t use it so (p.grad.data + group['lamda'] * (p.data - localweight.data)) is gradient of h_i