Add per batch Callback method

[nicofirst1]

Is your proposal related to a problem?

It would be nice to have to option to log metrics as the epoch proceeds during the training. this is useful for those epochs which requires hours to complete.

Describe the solution you'd like to have implemented

Maybe adding something of this kind after this line

        for callback in self.callbacks:
            callback.on_batch(self, interaction, optimized_loss, n_batches)

And add it to the default callback as:

class Callback:
...
def on_batch(self, interaction, optimized_loss, n_batches):  ...

[robertodessi]

I don't see why not (hoping it doesn't slow down training if we have many callbacks but I'm also assuming most callbacks wouldn't use it).

However, what is the use case exactly? Are you interested in batch statistics? I don't think they can be too informative given that in EGG mean loss is optimized (see e.g. here) and anything in aux is also averaged across batches when you use the default logging callback here

If it's to monitor training progress then I think other solutions might be better and more user-friendly #157

[nicofirst1]

Well yes I'm interested in both my specific loss and as well as some interaction on batch level. Since a batch to me takes an hour or so it would be really useful. Plus the callback can be used for a progress bar for #157

[robertodessi]

Regarding using the callback as a progress bar I believe it's a bit of a hacky/convoluted solution. Regarding batch-level loss and stats I don't see the callback to be extremely useful but don't have big reasons to reject either. I'll think about it but if @eugene-kharitonov is ok with it I guess we could have it

[nicofirst1]

Idk if there is a way to have both progress bar and loss/accuracy printing with the same function/method. Callback seems to me as the closes thing to use

[eugene-kharitonov]

Sounds good! What is n_batches in your snippet? I think Callback instance can count/reset them itself.

[eugene-kharitonov]

Also I like the use of it for the progress bar =) Hacky, but in a good, modular way ❤️

[robertodessi]

181 and https://github.com/facebookresearch/EGG/commit/88f73b9e684f8ef483c4e0d0d93fa7834fc87ce9 added this

[nicofirst1]

I think we should use the on_batch_end in the eval too. And in this case should there be two different methods? One for the training and one for the evaluation?

[nicofirst1]

It is probably better to pass a bool to the on_batch_end which is true for the training and false for the evaluation