search

danieltan07 / learning-to-reweight-examples

PyTorch Implementation of the paper Learning to Reweight Examples for Robust Deep Learning
351 stars 61 forks source link

How to add weight_decay and momentum when using this MetaModule? #11

Open KaiqiJinWow opened 5 years ago

KaiqiJinWow commented 5 years ago

I try to add weight_decay and momentum in the update_params() function in the model.py but I cannot figure it out yet. Could anybody give me some advice? Thank you in advance.

hansbu commented 5 years ago

weight_decay and momentum are optional arguments of the optim. Try to add them in the below code (in the jpynb file). opt = torch.optim.SGD(net.params(),lr=hyperparameters["lr"]) --> opt = torch.optim.SGD(net.params(),lr=hyperparameters["lr"], weight_decay=1e-4, momentum=0.9)