search

vllm-project / vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://docs.vllm.ai
Apache License 2.0
31.26k stars 4.75k forks source link

[New Model]: FM9GForCausalLM #8070

Open Aiwenqiuyu opened 3 months ago

Aiwenqiuyu commented 3 months ago

Your current environment

my vllm version is pip show vllm Name: vllm Version: 0.3.3+git3380931.abi0.dtk2404.torch2.1 Summary: A high-throughput and memory-efficient inference and serving engine for LLMs Home-page: https://github.com/vllm-project/vllm Author: vLLM Team Author-email: License: Apache 2.0 Location: /opt/conda/lib/python3.10/site-packages Requires: fastapi, ninja, numpy, prometheus-client, psutil, pydantic, ray, sentencepiece, starlette, tokenizers, transformers, typing-extensions, uvicorn Required-by:

🐛 Describe the bug

[Bug]: ValueError: Model architectures ['FM9GForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'FalconForCausalLM', 'GemmaForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QuantMixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'OLMoForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'PhiForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'RWForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM']

Before submitting a new issue...

DarkLight1337 commented 3 months ago

This model isn't supported in vLLM yet. Which HuggingFace repo is this from?

You are welcome to implement it and open a PR!

github-actions[bot] commented 2 days ago

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!