Is it possible to combine infobatch with the accelerate library?

NUS-HPC-AI-Lab / InfoBatch

Lossless Training Speed Up by Unbiased Dynamic Data Pruning

318 stars 18 forks source link

Is it possible to combine infobatch with the accelerate library? #21

Open zero0kiriyu opened 6 months ago

henryqin1997 commented 5 months ago

According to the accelerate usage case, I think it is possible to overload the original loss just before accelerator.backward(loss). We will make a test.