yangjianxin1 / Firefly

Firefly: 大模型训练工具,支持训练Qwen2.5、Qwen2、Yi1.5、Phi-3、Llama3、Gemma、MiniCPM、Yi、Deepseek、Orion、Xverse、Mixtral-8x7B、Zephyr、Mistral、Baichuan2、Llma2、Llama、Qwen、Baichuan、ChatGLM2、InternLM、Ziya2、Vicuna、Bloom等大模型
5.89k stars 526 forks source link

deepspeed安装报错 #255

Open TC10127 opened 6 months ago

TC10127 commented 6 months ago

(unsloth_env) D:\work\Firefly-master\Firefly-master>pip install deepspeed==0.9.5 Collecting deepspeed==0.9.5 Using cached deepspeed-0.9.5.tar.gz (809 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [15 lines of output] test.c LINK : fatal error LNK1181: 无法打开输入文件“aio.lib” Traceback (most recent call last): File "", line 2, in File "", line 34, in File "C:\Users\Administrator\AppData\Local\Temp\pip-install-zaj2fo7g\deepspeed_c6b12afd0dc34f24995fde18f2ea2bfe\setup.py", line 163, in abort(f"Unable to pre-compile {op_name}") File "C:\Users\Administrator\AppData\Local\Temp\pip-install-zaj2fo7g\deepspeed_c6b12afd0dc34f24995fde18f2ea2bfe\setup.py", line 51, in abort assert False, msg AssertionError: Unable to pre-compile async_io DS_BUILD_OPS=1 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] One can disable async_io with DS_BUILD_AIO=0 [ERROR] Unable to pre-compile async_io [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

pytorch=2.3,cuda=12.1

Parasolation commented 6 months ago

试试用conda安装呢