If we pull out the step loop from the epoch loop as a variable function, this will make the library compatible with closure-based optimizers or other optimizers (such as SAM).
To do this, we should add three versions
(default) regular steps
closure-based steps
two-step (like SAM)
Which version should be controlled by the Controller. We can create an enum or use a string.
If we pull out the step loop from the epoch loop as a variable function, this will make the library compatible with closure-based optimizers or other optimizers (such as SAM).
To do this, we should add three versions
Which version should be controlled by the Controller. We can create an enum or use a string.