[BUG] <title>AttributeError: 'QWenConfig' object has no attribute 'intermediate_size'

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

在纯python的环境下，执行CUDA_VISIBLE_DEVICES=1 python -m qanything_kernel.qanything_server.sanic_api --host 0.0.0.0 --port 14009 --model_size 7B，报错和vllm库有关系，去安装了vllm库之后报错： [ERROR] Experienced exception while trying to serve Traceback (most recent call last): File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/mixins/startup.py", line 958, in serve_single worker_serve(monitor_publisher=None, kwargs) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/worker/serve.py", line 143, in worker_serve raise e File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/worker/serve.py", line 117, in worker_serve return _serve_http_1( File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/server/runners.py", line 223, in _serve_http_1 loop.run_until_complete(app._server_event("init", "before")) File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/app.py", line 1764, in _server_event await self.dispatch( File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/signals.py", line 208, in dispatch return await dispatch File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/signals.py", line 183, in _dispatch raise e File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/signals.py", line 167, in _dispatch retval = await maybe_coroutine File "/workspace/envs/qanything-python/lib/python3.10/site-packages/sanic/app.py", line 1315, in _listener await maybe_coro File "/workspace/QAnything/qanything_kernel/qanything_server/sanic_api.py", line 202, in init_local_doc_qa local_doc_qa.init_cfg(args=args) File "/workspace/QAnything/qanything_kernel/core/local_doc_qa.py", line 61, in init_cfg self.llm: OpenAICustomLLM = OpenAICustomLLM(args) File "/workspace/QAnything/qanything_kernel/connector/llm/llm_for_fastchat.py", line 40, in init self.engine = AsyncLLMEngine.from_engine_args(engine_args) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 628, in from_engine_args engine = cls(parallel_config.worker_use_ray, File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 321, in init self.engine = self._init_engine(*args, *kwargs) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 369, in _init_engine return engine_class(args, kwargs) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 128, in init self._init_workers() File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 181, in _init_workers self._run_workers("load_model") File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 1041, in _run_workers driver_worker_output = getattr(self.driver_worker, File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/worker/worker.py", line 100, in load_model self.model_runner.load_model() File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 88, in load_model self.model = get_model(self.model_config, File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/model_executor/utils.py", line 52, in get_model return get_model_fn(model_config, device_config, **kwargs) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/model_executor/model_loader.py", line 79, in get_model model = model_class(model_config.hf_config, linear_method) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/model_executor/models/qwen.py", line 231, in init self.transformer = QWenModel(config, linear_method) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/model_executor/models/qwen.py", line 193, in init self.h = nn.ModuleList([ File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/model_executor/models/qwen.py", line 194, in QWenBlock(config, linear_method) File "/workspace/envs/qanything-python/lib/python3.10/site-packages/vllm/model_executor/models/qwen.py", line 148, in init config.intermediate_size // 2, File "/workspace/envs/qanything-python/lib/python3.10/site-packages/transformers/configuration_utils.py", line 264, in getattribute return super().getattribute(key) AttributeError: 'QWenConfig' object has no attribute 'intermediate_size'

看您requirement里边，vllm注释掉了0.3.3版本，又去下载了指定的vllm0.3.3版本，依然报错这个

期望行为 | Expected Behavior

No response

运行环境 | Environment

- OS:
- NVIDIA Driver:
- CUDA:
- docker:
- docker-compose:
- NVIDIA GPU:NVIDIA RTX A6000
- NVIDIA GPU Memory:48GB

QAnything日志 | QAnything logs

No response

复现方法 | Steps To Reproduce

执行CUDA_VISIBLE_DEVICES=1 python -m qanything_kernel.qanything_server.sanic_api --host 0.0.0.0 --port 14009 --model_size 7B

报错 AttributeError: 'QWenConfig' object has no attribute 'intermediate_size'

备注 | Anything else?

No response

netease-youdao / QAnything