search

luyug / GradCache

Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint
Apache License 2.0
327 stars 19 forks source link

Compatibility with Huggingface Trainer #2

Closed sh0416 closed 2 years ago

sh0416 commented 2 years ago

Hi,

First, congratulation of your nice and clear work.

I just wonder this code could be used with huggingface trainer.

I think it is bit tricky..

Thanks!

luyug commented 2 years ago

It should work (as long as you stay with Pytorch.)

In my works, I sub-class Trainer and override __init__ and training_step.

sh0416 commented 2 years ago

Thanks for the reply :) I will try that approach.