ofirpress / attention_with_linear_biases

Code for the ALiBi method for transformer language models (ICLR 2022)
MIT License
505 stars 39 forks source link

Is there any easy way to get a HF compatible version of your checkpoints? #12

Closed petroskarypis closed 1 year ago

ofirpress commented 1 year ago

MPT-7B uses ALiBi: https://huggingface.co/mosaicml/mpt-7b BLOOM does as well and so do the REPLIT LMs which I think are all on HF

petroskarypis commented 1 year ago

Thanks!