so we have gradient checkpointing enabled - and we have evaluation enabled in the same time.
So when huggingface trainer loop do evaluation - it switch model to model.eval mode and use torch,no_grad() or something like so, so input tensor does not need gradients.
Assume we have the following training script structure (copy-pasted my script):
Don't pay much attention to classes / models, they all are subclasses of standard transformers stuff with minimal changes,
The key issue is here:
so we have gradient checkpointing enabled - and we have evaluation enabled in the same time.
So when huggingface trainer loop do evaluation - it switch model to
model.eval
mode and usetorch,no_grad()
or something like so, so input tensor does not need gradients.Which cause the following code to make warnings:
warnings like this:
So I suggest to replace this code with the following one: