search

microsoft / Samba

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"
https://arxiv.org/pdf/2406.07522
MIT License
802 stars 48 forks source link

Support for Transformers library #12

Open HansPolo113 opened 4 months ago

HansPolo113 commented 4 months ago

Hi! Thank you for the great work on Samba! The hybrid model is very interesting.

Since Mamba-1 is now supported in the latest version of transformers, is there any plan for Samba to support the transformers library as well? This could benefit many researchers in the community.

Supporting transformers would allow for easier integration with existing workflows and tools, potentially increasing adoption and enabling more researchers to experiment with Samba.

renll commented 4 months ago

Thanks for the pointer! We will support the transformers libray when we release the 3.8B model.