Closed JoyHuYY1412 closed 3 years ago
Hi, @JoyHuYY1412, just a average over all the tasks. No we havent used any normalizations.
Hi, @JoyHuYY1412, just a average over all the tasks. No we havent used any normalizations.
See
Thank you for your reply. So if different tasks have parameters of different scales, e.g, A>>B, it seems the averaged network will be biased toward A. So when we try to recover the network for task B, we assume the memory samples of B can help fit? I don't know do I understand correctly.
Yes, that's one reason the classifier for a task is trained only in the inner loop. Also, to minimize the biased model we take a weighted average of the weights as we progress. Yes, finetuning with the memory samples helps to get a better model.
Yes, that's one reason the classifier for a task is trained only in the inner loop. Also, to minimize the biased model we take a weighted average of the weights as we progress. Thank you so much. I have two more questions.
- I read the pseudo code (algorithm 1) in your paper, so after we update phi in line 14, does the theta used in line 7 for task 1 is initialized from the updated phi ? and then task 2 initialized its theta from task 1?
- If so, does this operation somehow relieve the imbalance between tasks? Since after each update in the outer loop, the backbone network is reset.
I am curious about how you get the sum of the parameters of networks of each task. Did you apply any normalization?