search

berlino / gated_linear_attention

MIT License
97 stars 2 forks source link

A Full LM class #4

Closed Cranial-XIX closed 10 months ago

Cranial-XIX commented 10 months ago

Hi,

Thanks for this great work! I wonder if you could provide a wrapper for a full language model class, like in Mamba and RetNet they have MambaLMHeadModel and RetNetDecoder. Thanks a lot!

berlino commented 10 months ago

Thanks for your interests!

Yes, we plan to add the model file soon.

berlino commented 10 months ago

I've added a model class in gla_model.py, let me if there is any further questions