search

microsoft / torchscale

Foundation Architecture for (M)LLMs
https://aka.ms/GeneralAI
MIT License
3.01k stars 202 forks source link

Checkpoint for RetNet #99

Open macsz opened 6 months ago

macsz commented 6 months ago

Hi,

is there a pre-trained checkpoint available for RetNet, and if not - are there any plans to make it available?

Thanks!

Related to https://github.com/microsoft/unilm/issues/1474