Assistance on training a new retention network model ?

Jamie-Stirling / RetNet

An implementation of "Retentive Network: A Successor to Transformer for Large Language Models"

MIT License

1.14k stars 99 forks source link

Assistance on training a new retention network model ? #25

Open risedangel opened 9 months ago

risedangel commented 9 months ago

Hello I am new to llm world and i cant seem to train a new retnet model. Is there a script that i can use ? or can you guide me through some resources? Thank you in advance