Open yitzhaklevi opened 10 months ago
The issue is due to the fact that state_dict['state']['_mechanic'] has tensor pointers as keys, those does not have the same addresses when re-initializing new Mechanic optimizer
(other optimizers e.g AdamW has the indexes as keys for the state)
Thanks for bringing this up! I'll take a look and make an update (or if you want to do so, please feel free to submit a PR).
Welcome, I actually fixed that (will submit a PR next week, ) but it seems that on Windows the issue does not reproduce. (tried on my local machine and it worked)
Any update on this?
The following simple script reproduces the issue: