Maybe create our own module base class, with an API that is similar to Keras', providing fit, fit_generator, predict etc (with similar method parameters).
Downside could be that having the whole training strategy separately allows to use completely different training strategies.
We definitely need a standard way to distinguish between forward and whatever is supposed to calculate the loss, between these can be different things, and also between forward and predict!
Maybe create our own module base class, with an API that is similar to Keras', providing fit, fit_generator, predict etc (with similar method parameters).
Downside could be that having the whole training strategy separately allows to use completely different training strategies.
We definitely need a standard way to distinguish between forward and whatever is supposed to calculate the loss, between these can be different things, and also between forward and predict!