Open bhavnicksm opened 6 months ago
The reason why Config classes were originally set was because I expected all the optimizers to have a lot of "features" or options to tune in the future which makes the constructors really long.
Imagine the complexity if each new optimizer has to mention 30+ arguments in their init signature. Sounds tough!
This issue is related to #62
Currently every optimizer comes with a
config
specifically for that optimizer that manages the hyperparameters for the optimizer.This is made because of the following reasons:
defaults
that are useful.This diverges from the PyTorch-esque way optimizers are where the classes need to be provided with full information on all the parameters.
This issue is to find a decent middle ground between the two -> Allow for immediate tiny changes in the Hyperparameters on the spot, with the object, while also allowing for inheritable parameters.
This adds a significant overhead and makes things a bit more complex, so it should ideally be abstracted out in the BaseConfig and BaseOptimizer, with all the utility functions that transform from one format to the other.