Closed petroskarypis closed 1 year ago
MPT-7B uses ALiBi: https://huggingface.co/mosaicml/mpt-7b BLOOM does as well and so do the REPLIT LMs which I think are all on HF
Thanks!
MPT-7B uses ALiBi: https://huggingface.co/mosaicml/mpt-7b BLOOM does as well and so do the REPLIT LMs which I think are all on HF