UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`.

[DJstepbystep]

could anyone tell me how to solve this problem?followd is the discrible of the probelm.

E:\Anaconda\envs\pytorch-CycleGAN-and-pix2pix\lib\site-packages\torch\optim\lr_scheduler.py:134: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

[AJAX6255]

You might try Rdutta solution on the bottom of this page: https://discuss.pytorch.org/t/optimizer-step-before-lr-scheduler-step-error-using-gradscaler/92930 Others have had this same problem before.

On Wed, Apr 20, 2022 at 8:21 AM Kokowaah @.***> wrote:

could anyone tell me how to solve this problem?followd is the discrible of the probelm.

E:\Anaconda\envs\pytorch-CycleGAN-and-pix2pix\lib\site-packages\torch\optim\lr_scheduler.py:134: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)

— Reply to this email directly, view it on GitHub https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/issues/1412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AI6H6KZ3OHEOIAZUEFSHEHTVGAOGNANCNFSM5T4M3MKA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[vinodrajendran001]

I solved this issue by the following steps

1. update the def update_learning_rate(self): in base_model.py as below

   def update_learning_rate(self):
     pass

2. Add def update_learning_rate(self): in pix2pix_model.py as below

   def update_learning_rate(self):
   lrd = self.opt.lr / self.opt.n_epochs_decay
   lr = self.old_lr - lrd
   for param_group in self.optimizer_D.param_groups:
       param_group['lr'] = lr
   for param_group in self.optimizer_G.param_groups:
       param_group['lr'] = lr
   print('update learning rate: %f -> %f' % (self.old_lr, lr))
   self.old_lr = lr

3. Define self.old_lr = opt.lr in pix2pix_model.py as below

   if self.isTrain:
   # define loss functions
   self.criterionGAN = networks.GANLoss(opt.gan_mode).to(self.device)
   self.criterionL1 = torch.nn.L1Loss()
   self.old_lr = opt.lr  # insert this line
   self.optimizer_G = torch.optim.Adam(self.netG.parameters(), lr=opt.lr, betas=(opt.beta1, 0.999))
   self.optimizer_D = torch.optim.Adam(self.netD.parameters(), lr=opt.lr, betas=(opt.beta1, 0.999))
   self.optimizers.append(self.optimizer_G)
   self.optimizers.append(self.optimizer_D)

4. Move the model.update_learning_rate() to last line in train.py as below ......... print('End of epoch %d / %d \t Time Taken: %d sec' % (epoch, opt.n_epochs + opt.n_epochs_decay, time.time() - epoch_start_time)) if epoch > opt.n_epochs: model.update_learning_rate() # update learning rates at the end of every epoch.

[Parth527]

In the script train.py, just put the "model.update_learning_rate()" at the end of the inner for loop, and it will work.

[Y-JIE-CREATOR]

This warning message warns us that if we call lr_scheduler.step() before optimizer.step() , PyTorch will skip the first value of the learning rate schedule. But in theory, our learning rate remains unchanged in the first 100 epochs, which means that if we ignore this warning, the learning rate will begin to decay after the completion of the 99th epoch, so I think that in 200 epochs , this impact is very small. So if you don't want to change, just let it go.