search

JulesBelveze / bert-squeeze

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡
https://julesbelveze.github.io/bert-squeeze/
78 stars 10 forks source link

Fix/soft distillation #3

Closed JulesBelveze closed 2 years ago

JulesBelveze commented 2 years ago

This PR aims at testing and fixing the soft-distillation procedure.