[Bad Case]: python=3.11无法运行openai_api_server.py

[exthirteen]

Description / 描述

File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/model_executor/models/init.py", line 170, in resolve_model_cls raise ValueError( ValueError: Model architectures ['MiniCPM3ForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'CohereForCausalLM', 'DbrxForCausalLM', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'DeepseekV2ForCausalLM', 'ExaoneForCausalLM', 'FalconForCausalLM', 'GemmaForCausalLM', 'Gemma2ForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'JAISLMHeadModel', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QuantMixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'MiniCPMForCausalLM', 'NemotronForCausalLM', 'OlmoForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'PersimmonForCausalLM', 'PhiForCausalLM', 'Phi3ForCausalLM', 'PhiMoEForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'Qwen2MoeForCausalLM', 'RWForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM', 'ArcticForCausalLM', 'XverseForCausalLM', 'Phi3SmallForCausalLM', 'MedusaModel', 'EAGLEModel', 'MLPSpeculatorPreTrainedModel', 'JambaForCausalLM', 'GraniteForCausalLM', 'MistralModel', 'Blip2ForConditionalGeneration', 'ChameleonForConditionalGeneration', 'FuyuForCausalLM', 'InternVLChatModel', 'LlavaForConditionalGeneration', 'LlavaNextForConditionalGeneration', 'MiniCPMV', 'PaliGemmaForConditionalGeneration', 'Phi3VForCausalLM', 'UltravoxModel', 'BartModel', 'BartForConditionalGeneration']

Case Explaination / 案例解释

No response

[exthirteen]

运行 demo/function_call/function_calling.py 也会有类似的问题出现：

(minicpm3) root@feiye:/home/feiye/MiniCPM3-4B-service/MiniCPM-main/demo/function_call# CUDA_VISIBLE_DEVICES=0 python function_calling.py WARNING 09-09 11:21:09 cuda.py:69] Detected different devices in the system: WARNING 09-09 11:21:09 cuda.py:69] NVIDIA GeForce RTX 4090 D WARNING 09-09 11:21:09 cuda.py:69] Tesla T4 WARNING 09-09 11:21:09 cuda.py:69] Please make sure to set CUDA_DEVICE_ORDER=PCI_BUS_ID to avoid unexpected behavior. INFO 09-09 11:21:10 llm_engine.py:213] Initializing an LLM engine (v0.6.0) with config: model='/home/feiye/Model/MiniCPM3-4B', speculative_config=None, tokenizer='/home/feiye/Model/MiniCPM3-4B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=32768, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=/home/feiye/Model/MiniCPM3-4B, use_v2_block_manager=False, num_scheduler_steps=1, enable_prefix_caching=False, use_async_output_proc=True) INFO 09-09 11:21:10 model_runner.py:915] Starting to load model /home/feiye/Model/MiniCPM3-4B... rank0: Traceback (most recent call last): rank0: File "/home/feiye/MiniCPM3-4B-service/MiniCPM-main/demo/function_call/function_calling.py", line 80, in rank0: llm = LLM(model_path, trust_remote_code=True)

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/entrypoints/llm.py", line 177, in init rank0: self.llm_engine = LLMEngine.from_engine_args(

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 538, in from_engine_args rank0: engine = cls(

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/engine/llm_engine.py", line 305, in init rank0: self.model_executor = executor_class(

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/executor/executor_base.py", line 47, in init

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/executor/gpu_executor.py", line 40, in _init_executor

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/worker/worker.py", line 182, in load_model

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/worker/model_runner.py", line 917, in load_model rank0: self.model = get_model(model_config=self.model_config,

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/model_executor/model_loader/init.py", line 19, in get_model rank0: return loader.load_model(model_config=model_config,

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/model_executor/model_loader/loader.py", line 341, in load_model rank0: model = _initialize_model(model_config, self.load_config,

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/model_executor/model_loader/loader.py", line 168, in _initialize_model rank0: modelclass, = get_model_architecture(model_config)

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/model_executor/model_loader/utils.py", line 31, in get_model_architecture rank0: return ModelRegistry.resolve_model_cls(architectures)

rank0: File "/root/anaconda3/envs/minicpm3/lib/python3.11/site-packages/vllm/model_executor/models/init.py", line 170, in resolve_model_cls rank0: raise ValueError( rank0: ValueError: Model architectures ['MiniCPM3ForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausalLM', 'BloomForCausalLM', 'ChatGLMModel', 'ChatGLMForConditionalGeneration', 'CohereForCausalLM', 'DbrxForCausalLM', 'DeciLMForCausalLM', 'DeepseekForCausalLM', 'DeepseekV2ForCausalLM', 'ExaoneForCausalLM', 'FalconForCausalLM', 'GemmaForCausalLM', 'Gemma2ForCausalLM', 'GPT2LMHeadModel', 'GPTBigCodeForCausalLM', 'GPTJForCausalLM', 'GPTNeoXForCausalLM', 'InternLMForCausalLM', 'InternLM2ForCausalLM', 'JAISLMHeadModel', 'LlamaForCausalLM', 'LLaMAForCausalLM', 'MistralForCausalLM', 'MixtralForCausalLM', 'QuantMixtralForCausalLM', 'MptForCausalLM', 'MPTForCausalLM', 'MiniCPMForCausalLM', 'NemotronForCausalLM', 'OlmoForCausalLM', 'OPTForCausalLM', 'OrionForCausalLM', 'PersimmonForCausalLM', 'PhiForCausalLM', 'Phi3ForCausalLM', 'PhiMoEForCausalLM', 'QWenLMHeadModel', 'Qwen2ForCausalLM', 'Qwen2MoeForCausalLM', 'RWForCausalLM', 'StableLMEpochForCausalLM', 'StableLmForCausalLM', 'Starcoder2ForCausalLM', 'ArcticForCausalLM', 'XverseForCausalLM', 'Phi3SmallForCausalLM', 'MedusaModel', 'EAGLEModel', 'MLPSpeculatorPreTrainedModel', 'JambaForCausalLM', 'GraniteForCausalLM', 'MistralModel', 'Blip2ForConditionalGeneration', 'ChameleonForConditionalGeneration', 'FuyuForCausalLM', 'InternVLChatModel', 'LlavaForConditionalGeneration', 'LlavaNextForConditionalGeneration', 'MiniCPMV', 'PaliGemmaForConditionalGeneration', 'Phi3VForCausalLM', 'UltravoxModel', 'BartModel', 'BartForConditionalGeneration']

[Cppowboy]

可以先尝试安装支持minicpm3的vllm

pip install git+https://github.com/OpenBMB/vllm.git@minicpm3

[exthirteen]

可以先尝试安装支持minicpm3的vllm

pip install git+https://github.com/OpenBMB/vllm.git@minicpm3

感谢

[seasoncool]

@Cppowboy 請教下您，我在安裝vllm的時候報錯了，不知該怎麼解決，錯誤代碼如下：

Building wheels for collected packages: vllm Building wheel for vllm (pyproject.toml) ... error error: subprocess-exited-with-error

× Building wheel for vllm (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [100 lines of output] fatal: not a git repository (or any of the parent directories): .git

:56: RuntimeWarning: Failed to get commit hash: Command '['git', 'rev-parse', 'HEAD']' returned non-zero exit status 128. running bdist_wheel running build running build_py running build_ext -- The CXX compiler identification is GNU 9.4.0 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Build type: RelWithDebInfo -- Target device: cuda -- Found Python: /home/drc-whlab/miniconda3/envs/minicpm3-4b/bin/python3.11 (found version "3.11.9") found components: Interpreter Development.Module Development.SABIModule -- Found python matching: /home/drc-whlab/miniconda3/envs/minicpm3-4b/bin/python3.11. -- Found CUDA: /usr (found version "10.1") -- The CUDA compiler identification is NVIDIA 10.1.243 -- Detecting CUDA compiler ABI info -- Detecting CUDA compiler ABI info - done -- Check for working CUDA compiler: /usr/bin/nvcc - skipped -- Detecting CUDA compile features -- Detecting CUDA compile features - done -- Found CUDAToolkit: /usr/include (found version "10.1.243") -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Caffe2: CUDA detected: 10.1 -- Caffe2: CUDA nvcc is: /usr/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr CMake Error at /tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:77 (message): PyTorch requires CUDA 11.0 or above. Call Stack (most recent call first): /tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include) /tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package) CMakeLists.txt:67 (find_package) -- Configuring incomplete, errors occurred! Traceback (most recent call last): File "/home/drc-whlab/miniconda3/envs/minicpm3-4b/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/home/drc-whlab/miniconda3/envs/minicpm3-4b/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/drc-whlab/miniconda3/envs/minicpm3-4b/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel return _build_backend().build_wheel(wheel_directory, config_settings, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 421, in build_wheel return self._build_with_temp_dir( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir self.run_setup() File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 318, in run_setup exec(code, locals()) File "", line 456, in File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 117, in setup return distutils.core.setup(**attrs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 184, in setup return run_commands(dist) ^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 200, in run_commands dist.run_commands() File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands self.run_command(cmd) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 950, in run_command super().run_command(command) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_command cmd_obj.run() File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/command/bdist_wheel.py", line 384, in run self.run_command("build") File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command self.distribution.run_command(command) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 950, in run_command super().run_command(command) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_command cmd_obj.run() File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command self.distribution.run_command(command) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/dist.py", line 950, in run_command super().run_command(command) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 973, in run_command cmd_obj.run() File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 98, in run _build_ext.run(self) File "/tmp/pip-build-env-5nmhs033/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run self.build_extensions() File "", line 219, in build_extensions File "", line 201, in configure File "/home/drc-whlab/miniconda3/envs/minicpm3-4b/lib/python3.11/subprocess.py", line 413, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cmake', '/home/drc-whlab/james/MiniCPM3-4B/vllm-minicpm3', '-G', 'Ninja', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/home/drc-whlab/james/MiniCPM3-4B/vllm-mi nicpm3/build/lib.linux-x86_64-cpython-311/vllm', '-DCMAKE_ARCHIVE_OUTPUT_DIRECTORY=build/temp.linux-x86_64-cpython-311', '-DVLLM_TARGET_DEVICE=cuda', '-DVLLM_PYTHON_EXECUTABLE=/home/drc-whlab/miniconda3/envs/minicpm3-4b/bin/python3.11 ', '-DCMAKE_JOB_POOL_COMPILE:STRING=compile', '-DCMAKE_JOB_POOLS:STRING=compile=20']' returned non-zero exit status 1. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for vllm Failed to build vllm ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (vllm)

[LDLINGLINGLING]

你好，我跟新了安装vllm失败如何解决的教程，https://modelbest.feishu.cn/wiki/LrdMwKKt3iZgoYkQlPRcvY1PnXc#share-Uqe2dAAdeozr9exchnlcXI6inTb