For instance, I have changed the self._network.parameters() to filter(lambda p: p.requires_grad, self._network.parameters( )). Note that if you do not use the filter function to drop those parameters that do not require updates, the weight decay in your optimizer will still change their value even though you have set them as requires_grad=False.
I printed out the mean value of prompt_pool[0].mean() when learning task 5, you can see that it was decreasing.
Hi, thanks for your repo.
I found some typos and bugs in the repo.
For instance, I have changed the
self._network.parameters()tofilter(lambda p: p.requires_grad, self._network.parameters( )). Note that if you do not use the filter function to drop those parameters that do not require updates, the weight decay in your optimizer will still change their value even though you have set them asrequires_grad=False.I printed out the mean value of
prompt_pool[0].mean()when learningtask 5, you can see that it was decreasing.