search

ModelTC / lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Apache License 2.0
2.66k stars 212 forks source link

[FEATURE] Add support for true open-source models like MPT #102

Open casper-hansen opened 1 year ago

casper-hansen commented 1 year ago

Add support for MPT models, which are licensed under Apache 2.0 just light LightLLM.

They have 7B and 30B with different context lengths, here is one of them: https://huggingface.co/mosaicml/mpt-7b-8k

shihaobai commented 1 year ago

@casperbh96 Thank for your attention. We will add this model.