bigscience-workshop / multilingual-modeling

BLOOM+1: Adapting BLOOM model to support a new unseen language
https://arxiv.org/abs/2212.09535
Apache License 2.0
69 stars 15 forks source link

[Lower Priority] Implement "Fisher Induced Sparse uncHanging (FISH) Mask" #38

Open haileyschoelkopf opened 2 years ago

haileyschoelkopf commented 2 years ago

Paper: https://arxiv.org/pdf/2111.09839.pdf

Code: https://github.com/varunnair18/FISH/blob/main/transformers/examples/text-classification/run_glue_sparse_update.py

This will be relatively easy to add to our code.