[Model Request] InternLM2.5-20B , 7B, and 1.8B

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation

https://llm.mlc.ai/

Apache License 2.0

19.08k stars 1.56k forks source link

[Model Request] InternLM2.5-20B , 7B, and 1.8B #2843

Closed Sing-Li closed 2 months ago

Sing-Li commented 2 months ago

⚙️ Request New Models

Link to an existing implementation (e.g. Hugging Face/Github): https://huggingface.co/internlm
Is this model architecture supported by MLC-LLM? (the list of supported models) No

Additional context

They seem to have their own quantized model assembly + deployment pipeline already but having it under MLC LLM pipeline will add significant value for all.

tlopex commented 2 months ago

Thanks for your issue! @Sing-Li I thought that we have already supported Internlm2.5 since it has the same model structure with Internlm2.

Sing-Li commented 2 months ago

Thanks @tlopex Indeed it looks like the model is fully supported here. https://github.com/mlc-ai/mlc-llm/pull/2630

BUT I was not able to find any ready-to-run converted model weights on huggingface ( https://huggingface.co/mlc-ai ) do you know of another source for them?

tlopex commented 2 months ago

@Sing-Li Thanks for pointing out that. I'll upload model weights to the huggingface repo later.

tlopex commented 2 months ago

Hi @Sing-Li , ready-to-run converted models of internlm2.5 with 1.8B, 7B and 20B have been uploaded to HF repo of mlc-ai, sorry for being late. Thanks for @CharlieFRuan @MasterJH5574 's help.