pip install deepspeed error

[luieswww]

[ERROR] Unable to pre-compile async_io

[Sleepychord]

请按照readme的办法跳过安装deepspeed，或者按照报错信息设置环境变量后装上deepspeed。

[nameless0704]

请按照readme的办法跳过安装deepspeed，或者按照报错信息设置环境变量后装上deepspeed。

可以跳过deepspeed安装，但是用huggingface调模型的时候还是需要SAT，然后SAT还是需要deepspeed...

[Sleepychord]

@nameless0704 sat依赖里有deepspeed，但是不装也能进行模型推理只是不能训练。请严格按照readme里的跳过步骤，肯定就没有问题了。

[earthxx]

LINK : fatal error LNK1181: 无法打开输入文件“aio.lib”

PermissionError: [WinError 5] 拒绝访问。: '.\deepspeed\ops\csrc' DS_BUILD_OPS=1 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] cpu_adagrad requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] cpu_adagrad requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] cpu_adam requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] cpu_adam requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.0 [WARNING] please install triton==1.0.0 if you want to use sparse attention Install Ops={'async_io': False, 'cpu_adagrad': 1, 'cpu_adam': 1, 'fused_adam': 1, 'fused_lamb': 1, 'quantizer': 1, 'random_ltd': 1, 'sparse_attn': False, 'spatial_inference': 1, 'transformer': 1, 'stochastic_transformer': 1, 'transformer_inference': 1, 'utils': 1}

不支持touch2.0？ 无法安装deepspeed

[bobo0810]

LINK : fatal error LNK1181: 无法打开输入文件“aio.lib”

PermissionError: [WinError 5] 拒绝访问。: '.\deepspeed\ops\csrc' DS_BUILD_OPS=1 [WARNING] async_io requires the dev libaio .so object and headers but these were not found. [WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found. [WARNING] cpu_adagrad requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] cpu_adagrad requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adagrad attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] cpu_adam requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] cpu_adam requires the 'lscpu' command, but it does not exist! [WARNING] cpu_adam attempted to query 'lscpu' after failing to use py-cpuinfo to detect the CPU architecture. 'lscpu' does not appear to exist on your system, will fall back to use -march=native and non-vectorized execution. [WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.0 [WARNING] please install triton==1.0.0 if you want to use sparse attention Install Ops={'async_io': False, 'cpu_adagrad': 1, 'cpu_adam': 1, 'fused_adam': 1, 'fused_lamb': 1, 'quantizer': 1, 'random_ltd': 1, 'sparse_attn': False, 'spatial_inference': 1, 'transformer': 1, 'stochastic_transformer': 1, 'transformer_inference': 1, 'utils': 1}

不支持touch2.0？ 无法安装deepspeed

https://github.com/THUDM/VisualGLM-6B/issues/17#issuecomment-1558406692 torch2.0+deepspeed我安装没问题，但运行有报错

[earthxx]

cu117? torch2.0的报错主要出现在waring中。 deepspeed 主要报错LINK : fatal error LNK1181: 无法打开输入文件“aio.lib”

[liudenghuiv]

pip install -i https://mirrors.aliyun.com/pypi/simple/ -r requirements.txt pip install -i https://mirrors.aliyun.com/pypi/simple/ -r requirements_wo_ds.txt pip install -i https://mirrors.aliyun.com/pypi/simple/ --no-deps "SwissArmyTransformer>=0.3.6" 绕过就行 别轴

[sunxiaoyu12]

我绕过了deepspeed，但是无法推理