search

Jannoshh / simple-sam

Sharpness-Aware Minimization for Efficiently Improving Generalization
MIT License
41 stars 9 forks source link

Is it possible to use this with huggingface transformers? #7

Closed karims closed 3 years ago

karims commented 3 years ago

Can I use this library with huggingface transformers? If so, is there any such example? I couldn't find it in the repo.

Jannoshh commented 3 years ago

No, the training loops used for the huggingface transformers are custom and optimized for their models.

From their GitHub: "The training API is not intended to work on any model but is optimized to work with the models provided by the library. For generic machine learning loops, you should use another library."

Implementing SAM for the huggingface transformers would need a high understanding of their implementation. Maybe you could ask the authors whether they've tried using SAM for their training loops?