search

JulesBelveze / bert-squeeze

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡
https://julesbelveze.github.io/bert-squeeze/
78 stars 10 forks source link

Examples of Joint Pruning, Quantization and Distillation #49

Closed g12bftd closed 1 year ago

g12bftd commented 1 year ago

Hey, Thank you for making this repo public.

Do you have any example scripts that combine pruning, quantization and distillation (ideally for BERT-Base models)?

JulesBelveze commented 1 year ago

Hey @g12bftd thanks for showing interest!

I am actually cooking up some tutorial notebooks right now! I will add one that combines pruning, quantization and distillation for you 🤓 I'll ping you once ready!

g12bftd commented 1 year ago

Hey @JulesBelveze, thank you so much! If you could show an example where teacher = Bert-large-uncased and student= bert-base-uncased on any of the glue tasks, it would be fantastic!

JulesBelveze commented 1 year ago

Hey @g12bftd could you to run this notebook? https://github.com/JulesBelveze/bert-squeeze/blob/docs/tutorials/docs/tutorials/Distillation.ipynb

Let me know if you run into any issues 😃

g12bftd commented 1 year ago

Hey @JulesBelveze, thank you for getting back so quickly. I am trying to run the notebook in Google Colab. Some of the required packages cannot be installed with poetry. Any advice?

JulesBelveze commented 1 year ago

@g12bftd Are you trying to install it from the docs/tutorials branch?

g12bftd commented 1 year ago

Hey @JulesBelveze, I've attached my full notebook in a zip file. You should be able to see the exact lines of code I execute as well as the corresponding error messages. KD and Quant BERT.ipynb.zip

JulesBelveze commented 1 year ago

@g12bftd Sweet thanks 😸 Can you try to replace:

!git clone https://github.com/JulesBelveze/bert-squeeze.git

by:

!git clone -b docs/tutorials https://github.com/JulesBelveze/bert-squeeze.git
g12bftd commented 1 year ago

Testing2.ipynb.zip Testing.ipynb.zip Still no luck. A different error message.

JulesBelveze commented 1 year ago

@g12bftd those don't seem related to bert-squeeze but more to hardware stuff 😅 Could you directly share with me your Colab's url??

g12bftd commented 1 year ago

@JulesBelveze yeah I figured. Here is the link: https://colab.research.google.com/drive/1zKcerP-LhcHSv2uM5lnou1yECptIjrvb?usp=sharing. Thanks again for your help!

JulesBelveze commented 1 year ago

Jeez I hate notebooks 🤣 I am fighting with the numpy installation.. There might be a compatibility issue with Python 3.10 and this version of numpy 🤔

JulesBelveze commented 1 year ago

@g12bftd can you try to install the latest version and check with it?