search

mlc-ai / mlc-llm

Universal LLM Deployment Engine with ML Compilation
https://llm.mlc.ai/
Apache License 2.0
19.27k stars 1.59k forks source link

[Model Request] Jiutian 13.9b MoE model #2935

Open 20241123 opened 2 months ago

20241123 commented 2 months ago

🐛 Bug

I am trying to work with Jiutian 13.9b MoE model.But getting error in model compilation step.

To Reproduce

Steps to reproduce the behavior:

1. pip install --pre -U -f https://mlc.ai/wheels mlc-chat-nightly-cu122 mlc-ai-nightly-cu122 git lfs install git clone https://huggingface.co/clemenschen/Jiutian-139MoE-chat-4bit mlc_llm convert_weight Jiutian-139MoE-chat --quantization q4f16_1 -o Jiutian-139MoE-chat_mlc

Results in:

raise ValueError(f"Unknown model type: {model_type}. Available ones: {list(MODELS.keys())}") ValueError: Unknown model type: jiutian. Available ones: ['llama', 'mistral', 'gemma', 'gemma2', 'gpt2', 'mixtral', 'gpt_neox', 'gpt_bigcode', 'phi-msft', 'phi', 'phi3', 'phi3_v', 'qwen', 'qwen2', 'qwen2_moe', 'stablelm', 'baichuan', 'internlm', 'internlm2', 'rwkv5', 'orion', 'llava', 'rwkv6', 'chatglm', 'eagle', 'bert', 'medusa', 'starcoder2', 'cohere', 'minicpm']

Expected behavior

Should convert the weights. It works fine with qwen and qwen2.

Environment

Operating system: Ubuntu Device: v100 How you installed MLC-LLM: pip AND source How you installed TVM-Unity: pip Python version: 3.10 GPU driver version (if applicable): 5.7

MasterJH5574 commented 2 months ago

Hi @20241123 thanks for the question. This model is in a new architecture that MLC does not yet support. Contributions are welcome :-)