feature(deebert): enable batch size > 1 at inference time

JulesBelveze / bert-squeeze

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡

https://julesbelveze.github.io/bert-squeeze/

78 stars 10 forks source link

Closed JulesBelveze closed 1 year ago

JulesBelveze commented 1 year ago

This PR aims at enabling inferencing DeeBert with a batch size > 1.