Thank you for your nice codes and it works quite well on MNIST reconstruction!
However, I do notice a very interesting phenomena: Although the training seems to be successful and the visualization of reconstruction looks good, the loss is increasing during training. But it seems that the optimizer in pytorch should always lower the training loss. I am very curious about the reason, and would you please explain why?
I also print the L1loss between the original images and the reconstructed images, and the L1loss is decreasing as expected.
Hi,
Thank you for your nice codes and it works quite well on MNIST reconstruction!
However, I do notice a very interesting phenomena: Although the training seems to be successful and the visualization of reconstruction looks good, the loss is increasing during training. But it seems that the optimizer in pytorch should always lower the training loss. I am very curious about the reason, and would you please explain why?
I also print the L1loss between the original images and the reconstructed images, and the L1loss is decreasing as expected.
Best regards, Wang