Open CasellaJr opened 2 years ago
Here is my analysis on the issue:
Issue Reproduction:
Issue is observed with torch>=1.8.0 but not with torch <=1.7.1 torch<=1.7.1
Logs:
INFO #### opt_state_dict {'state': {}, 'param_groups': [{'lr': 0.1, 'momentum': 0, 'dampening': 0, 'weight_decay': 0, pytorch_adapter.py:121
'nesterov': False, 'maximize': False, 'foreach': None, 'differentiable': False, 'params': []}]} ####
WARNING tried to remove tensor: __opt_state_needed not present in the tensor dict utils.py:170
INFO #### opt_state_dict {'state': {}, 'param_groups': [{'lr': 0.1, 'momentum': 0, 'dampening': 0, 'weight_decay': 0, pytorch_adapter.py:121
'nesterov': False, 'maximize': False, 'foreach': None, 'differentiable': False, 'params': []}]} ####
torch>=1.8.0
Logs:
INFO #### opt_state_dict {'state': {0: {'momentum_buffer': None}, 1: {'momentum_buffer': None}}, 'param_groups': [{'lr': pytorch_adapter.py:120
0.0001, 'momentum': 0, 'dampening': 0, 'weight_decay': 0, 'nesterov': False, 'maximize': False, 'foreach': None,
'differentiable': False, 'params': [0, 1]}]} ####
WARNING tried to remove tensor: __opt_state_needed not present in the tensor dict utils.py:172
INFO #### opt_state_dict {'state': {0: {'momentum_buffer': None}, 1: {'momentum_buffer': None}}, 'param_groups': [{'lr': pytorch_adapter.py:120
0.0001, 'momentum': 0, 'dampening': 0, 'weight_decay': 0, 'nesterov': False, 'maximize': False, 'foreach': None,
'differentiable': False, 'params': [0, 1]}]} ####
Additional Information Issue not observed if momentum!=0
Next Steps: I will raise a PR for fixing this issue.
Thank you for the answer. So, until now we can fix only by changing the momentum?
Yes, for now we can fix this by giving a non-zero value to momentum
Hello. I have this error below when using SGD as optimizer. With Adam it work correctly.
Basically in this setting it work, but if I comment Adam and uncomment SGD then I have this error: