modelscope / data-juicer

Making data higher-quality, juicier, and more digestible for any large models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据!
Apache License 2.0
3.02k stars 181 forks source link

编译安装时报错 #486

Open charonkk opened 2 weeks ago

charonkk commented 2 weeks ago

Before Asking 在提问之前

Search before asking 先搜索,再提问

Question

基于NPU编译安装时出现以下问题,是否是设备问题? WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Installing build dependencies ... done Running command Getting requirements to build wheel /tmp/pip-build-env-suyjv95d/overlay/lib/python3.10/site-packages/torch/_subclasses/functional_tensor.py:258: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:84.) cpu = _conversion_method_template(device=torch.device("cpu")) Traceback (most recent call last): File "/root/miniconda3/envs/datajuicer/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/root/miniconda3/envs/datajuicer/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/root/miniconda3/envs/datajuicer/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel return hook(config_settings) File "/tmp/pip-build-env-suyjv95d/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel return self._get_build_requires(config_settings, requirements=[]) File "/tmp/pip-build-env-suyjv95d/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires self.run_setup() File "/tmp/pip-build-env-suyjv95d/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in run_setup exec(code, locals()) File "", line 487, in File "", line 394, in get_vllm_version RuntimeError: Unknown runtime environment error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: /root/miniconda3/envs/datajuicer/bin/python /root/miniconda3/envs/datajuicer/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpf0wp8kc5 cwd: /tmp/pip-install-iwibg5im/vllm_1db146cd6afa4d3cb15da9a4ea4713ac Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Additional 额外信息

No response

drcege commented 1 week ago

看起来是 torchvllm 与 NPU 的兼容性问题;可以尝试手动安装这些包,然后在 science_requires.txt 注释掉后再安装 data-juicer。

另外,如果不需要 vllm 推理,可以跳过安装这个包。