stochasticai / xTuring

Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
https://xturing.stochastic.ai
Apache License 2.0
2.61k stars 207 forks source link

Add Mamba to available LLMs #284

Closed mapmeld closed 1 month ago

mapmeld commented 1 month ago

Summary

This PR adds support for Mamba models (by default mamba-2.8b) which are an alternative to attention-based LLMs.

Changes include:

Notes:

Checklist

Demonstrated inference in this notebook: https://colab.research.google.com/drive/1-i4xmsyppWBdwR1qt6QN21m0uiXu0guM?usp=sharing

MarcosRiveraMartinez commented 1 month ago

@mapmeld Thanks for your contribution!