search

microsoft / DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
https://www.deepspeed.ai/
Apache License 2.0
35.29k stars 4.09k forks source link

Installation on Windows 10 (Deepspeed inference) #2588

Closed Eichhof closed 2 months ago

Eichhof commented 1 year ago

Hello

I'm running Windows 10 and I would like to install DeepSpeed to speed up inference of GPT-J. My system is the following:

Windows 10
cuda 11.6
torch 1.13.0
Python 3.9.12

When running the following command in Powershell to build the wheel

$env:DS_BUILD_OPS = 0
python setup.py bdist_wheel

I'm getting the following error (same when running pip install deepspeed):

DS_BUILD_OPS=0
←[93m [WARNING] ←[0m please install triton==1.0.0 if you want to use sparse attention
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
←[93m [WARNING] ←[0m async_io requires the dev libaio .so object and headers but these were not found.
←[93m [WARNING] ←[0m If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
Install Ops={'cpu_adam': False, 'cpu_adagrad': False, 'fused_adam': False, 'fused_lamb': False, 'sparse_attn': False, 'transformer': False, 'stochastic_transformer': False, 'async_io': False, 'utils': False, 'quantizer': False, 'transformer_inference': False, 'spatial_inference': False}
Traceback (most recent call last):
  File "C:\Users\myUsername\Desktop\DeepSpeed\setup.py", line 196, in <module>
    create_dir_symlink('..\\..\\csrc', '.\\deepspeed\\ops\\csrc')
  File "C:\Users\myUsername\Desktop\DeepSpeed\setup.py", line 190, in create_dir_symlink
    os.symlink(src, dest)
OSError: [WinError 1314] A required privilege is not held by the client: '..\\..\\csrc' -> '.\\deepspeed\\ops\\csrc'

Can somebody help me getting inference with DeepSpeed running on Windows?

RezaYazdaniAminabadi commented 1 year ago

Hi @Eichhof,

Can you please confirm if you are running this in administrative mode?

Thanks, Reza

Eichhof commented 1 year ago

Thank you very much. In administrator mode I could now build the wheel. I then installed the wheel using pip. The problem now is that when calling deepspeed.init_inference(...), I'm getting the following error:

FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\myUsername\Anaconda3\envs\huggingface\lib\site-packages\deepspeed\ops\csrc/transformer/inference/csrc/pt_binding.cpp'

In fact, no folder csrcexists in ops.

If I set $env:DS_BUILD_OPS = 1, building wheel fails due to missing triton 1.0.0 but triton is not available on Windows. If I then set $env:DS_BUILD_SPARSE_ATTN = 0, I'm getting the error LINK : fatal error LNK1181: cannot open input file 'aio.lib'. If I then set $env:DS_BUILD_AIO = 0, I'm getting the following error:

DS_BUILD_OPS=1
 [WARNING]  please install triton==1.0.0 if you want to use sparse attention
test.c
LINK : fatal error LNK1181: cannot open input file 'aio.lib'
 [WARNING]  async_io requires the dev libaio .so object and headers but these were not found.
 [WARNING]  If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
Install Ops={'cpu_adam': 1, 'cpu_adagrad': 1, 'fused_adam': 1, 'fused_lamb': 1, 'sparse_attn': False, 'transformer': 1, 'stochastic_transformer': 1, 'async_io': False, 'utils': 1, 'quantizer': 1, 'transformer_inference': 1, 'spatial_inference': 1}
version=0.7.7+18713c68, git_hash=18713c68, git_branch=master
install_requires=['hjson', 'ninja', 'numpy', 'packaging', 'psutil', 'py-cpuinfo', 'pydantic', 'torch', 'tqdm']
compatible_ops={'cpu_adam': True, 'cpu_adagrad': True, 'fused_adam': True, 'fused_lamb': True, 'sparse_attn': False, 'transformer': True, 'stochastic_transformer': True, 'async_io': False, 'utils': True, 'quantizer': True, 'transformer_inference': True, 'spatial_inference': True}
ext_modules=[<setuptools.extension.Extension('deepspeed.ops.adam.cpu_adam_op') at 0x27a5a19e680>, <setuptools.extension.Extension('deepspeed.ops.adagrad.cpu_adagrad_op') at 0x27a63662aa0>, <setuptools.extension.Extension('deepspeed.ops.adam.fused_adam_op') at 0x27a65123dc0>, <setuptools.extension.Extension('deepspeed.ops.lamb.fused_lamb_op') at 0x27a65123fa0>, <setuptools.extension.Extension('deepspeed.ops.transformer.transformer_op') at 0x27a65123e20>, <setuptools.extension.Extension('deepspeed.ops.transformer.stochastic_transformer_op') at 0x27a65219cf0>, <setuptools.extension.Extension('deepspeed.ops.utils_op') at 0x27a65219f30>, <setuptools.extension.Extension('deepspeed.ops.quantizer.quantizer_op') at 0x27a6521a1a0>, <setuptools.extension.Extension('deepspeed.ops.transformer.inference.transformer_inference_op') at 0x27a6521a050>, <setuptools.extension.Extension('deepspeed.ops.spatial.spatial_inference_op') at 0x27a6521a230>]
running bdist_wheel
running build
running build_py
creating build\lib.win-amd64-3.10
creating build\lib.win-amd64-3.10\benchmarks
copying benchmarks\__init__.py -> build\lib.win-amd64-3.10\benchmarks
creating build\lib.win-amd64-3.10\deepspeed
copying deepspeed\constants.py -> build\lib.win-amd64-3.10\deepspeed
copying deepspeed\env_report.py -> build\lib.win-amd64-3.10\deepspeed
copying deepspeed\git_version_info.py -> build\lib.win-amd64-3.10\deepspeed
copying deepspeed\git_version_info_installed.py -> build\lib.win-amd64-3.10\deepspeed
copying deepspeed\__init__.py -> build\lib.win-amd64-3.10\deepspeed
creating build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\all_gather.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\all_reduce.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\all_to_all.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\broadcast.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\constants.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\pt2pt.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\run_all.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\utils.py -> build\lib.win-amd64-3.10\benchmarks\communication
copying benchmarks\communication\__init__.py -> build\lib.win-amd64-3.10\benchmarks\communication
creating build\lib.win-amd64-3.10\deepspeed\accelerator
copying deepspeed\accelerator\abstract_accelerator.py -> build\lib.win-amd64-3.10\deepspeed\accelerator
copying deepspeed\accelerator\cuda_accelerator.py -> build\lib.win-amd64-3.10\deepspeed\accelerator
copying deepspeed\accelerator\real_accelerator.py -> build\lib.win-amd64-3.10\deepspeed\accelerator
copying deepspeed\accelerator\__init__.py -> build\lib.win-amd64-3.10\deepspeed\accelerator
creating build\lib.win-amd64-3.10\deepspeed\autotuning
copying deepspeed\autotuning\autotuner.py -> build\lib.win-amd64-3.10\deepspeed\autotuning
copying deepspeed\autotuning\config.py -> build\lib.win-amd64-3.10\deepspeed\autotuning
copying deepspeed\autotuning\constants.py -> build\lib.win-amd64-3.10\deepspeed\autotuning
copying deepspeed\autotuning\scheduler.py -> build\lib.win-amd64-3.10\deepspeed\autotuning
copying deepspeed\autotuning\utils.py -> build\lib.win-amd64-3.10\deepspeed\autotuning
copying deepspeed\autotuning\__init__.py -> build\lib.win-amd64-3.10\deepspeed\autotuning
creating build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\constants.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\deepspeed_checkpoint.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\reshape_3d_utils.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\reshape_meg_2d.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\reshape_utils.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\universal_checkpoint.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\utils.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\zero_checkpoint.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
copying deepspeed\checkpoint\__init__.py -> build\lib.win-amd64-3.10\deepspeed\checkpoint
creating build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\backend.py -> build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\comm.py -> build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\config.py -> build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\constants.py -> build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\torch.py -> build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\utils.py -> build\lib.win-amd64-3.10\deepspeed\comm
copying deepspeed\comm\__init__.py -> build\lib.win-amd64-3.10\deepspeed\comm
creating build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\basic_layer.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\compress.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\config.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\constants.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\helper.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\scheduler.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\utils.py -> build\lib.win-amd64-3.10\deepspeed\compression
copying deepspeed\compression\__init__.py -> build\lib.win-amd64-3.10\deepspeed\compression
creating build\lib.win-amd64-3.10\deepspeed\elasticity
copying deepspeed\elasticity\config.py -> build\lib.win-amd64-3.10\deepspeed\elasticity
copying deepspeed\elasticity\constants.py -> build\lib.win-amd64-3.10\deepspeed\elasticity
copying deepspeed\elasticity\elasticity.py -> build\lib.win-amd64-3.10\deepspeed\elasticity
copying deepspeed\elasticity\elastic_agent.py -> build\lib.win-amd64-3.10\deepspeed\elasticity
copying deepspeed\elasticity\utils.py -> build\lib.win-amd64-3.10\deepspeed\elasticity
copying deepspeed\elasticity\__init__.py -> build\lib.win-amd64-3.10\deepspeed\elasticity
creating build\lib.win-amd64-3.10\deepspeed\inference
copying deepspeed\inference\config.py -> build\lib.win-amd64-3.10\deepspeed\inference
copying deepspeed\inference\engine.py -> build\lib.win-amd64-3.10\deepspeed\inference
copying deepspeed\inference\__init__.py -> build\lib.win-amd64-3.10\deepspeed\inference
creating build\lib.win-amd64-3.10\deepspeed\launcher
copying deepspeed\launcher\constants.py -> build\lib.win-amd64-3.10\deepspeed\launcher
copying deepspeed\launcher\launch.py -> build\lib.win-amd64-3.10\deepspeed\launcher
copying deepspeed\launcher\multinode_runner.py -> build\lib.win-amd64-3.10\deepspeed\launcher
copying deepspeed\launcher\runner.py -> build\lib.win-amd64-3.10\deepspeed\launcher
copying deepspeed\launcher\__init__.py -> build\lib.win-amd64-3.10\deepspeed\launcher
creating build\lib.win-amd64-3.10\deepspeed\model_implementations
copying deepspeed\model_implementations\__init__.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations
creating build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\inject.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\layers.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\load_checkpoint.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\module_quantize.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\replace_module.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\replace_policy.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
copying deepspeed\module_inject\__init__.py -> build\lib.win-amd64-3.10\deepspeed\module_inject
creating build\lib.win-amd64-3.10\deepspeed\moe
copying deepspeed\moe\experts.py -> build\lib.win-amd64-3.10\deepspeed\moe
copying deepspeed\moe\layer.py -> build\lib.win-amd64-3.10\deepspeed\moe
copying deepspeed\moe\mappings.py -> build\lib.win-amd64-3.10\deepspeed\moe
copying deepspeed\moe\sharded_moe.py -> build\lib.win-amd64-3.10\deepspeed\moe
copying deepspeed\moe\utils.py -> build\lib.win-amd64-3.10\deepspeed\moe
copying deepspeed\moe\__init__.py -> build\lib.win-amd64-3.10\deepspeed\moe
creating build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\config.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\constants.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\csv_monitor.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\monitor.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\tensorboard.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\utils.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\wandb.py -> build\lib.win-amd64-3.10\deepspeed\monitor
copying deepspeed\monitor\__init__.py -> build\lib.win-amd64-3.10\deepspeed\monitor
creating build\lib.win-amd64-3.10\deepspeed\nebula
copying deepspeed\nebula\config.py -> build\lib.win-amd64-3.10\deepspeed\nebula
copying deepspeed\nebula\constants.py -> build\lib.win-amd64-3.10\deepspeed\nebula
copying deepspeed\nebula\__init__.py -> build\lib.win-amd64-3.10\deepspeed\nebula
creating build\lib.win-amd64-3.10\deepspeed\ops
copying deepspeed\ops\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops
creating build\lib.win-amd64-3.10\deepspeed\pipe
copying deepspeed\pipe\__init__.py -> build\lib.win-amd64-3.10\deepspeed\pipe
creating build\lib.win-amd64-3.10\deepspeed\profiling
copying deepspeed\profiling\config.py -> build\lib.win-amd64-3.10\deepspeed\profiling
copying deepspeed\profiling\constants.py -> build\lib.win-amd64-3.10\deepspeed\profiling
copying deepspeed\profiling\__init__.py -> build\lib.win-amd64-3.10\deepspeed\profiling
creating build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\bf16_optimizer.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\config.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\config_utils.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\constants.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\dataloader.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\eigenvalue.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\engine.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\lr_schedules.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\progressive_layer_drop.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\quantize.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\sparse_tensor.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\state_dict_factory.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\utils.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\weight_quantizer.py -> build\lib.win-amd64-3.10\deepspeed\runtime
copying deepspeed\runtime\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime
creating build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\comms_logging.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\debug.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\exceptions.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\groups.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\init_on_device.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\logging.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\mixed_precision_linkage.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\nvtx.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\tensor_fragment.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\timer.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\types.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\zero_to_fp32.py -> build\lib.win-amd64-3.10\deepspeed\utils
copying deepspeed\utils\__init__.py -> build\lib.win-amd64-3.10\deepspeed\utils
creating build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
copying deepspeed\autotuning\tuner\base_tuner.py -> build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
copying deepspeed\autotuning\tuner\cost_model.py -> build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
copying deepspeed\autotuning\tuner\index_based_tuner.py -> build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
copying deepspeed\autotuning\tuner\model_based_tuner.py -> build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
copying deepspeed\autotuning\tuner\utils.py -> build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
copying deepspeed\autotuning\tuner\__init__.py -> build\lib.win-amd64-3.10\deepspeed\autotuning\tuner
creating build\lib.win-amd64-3.10\deepspeed\model_implementations\diffusers
copying deepspeed\model_implementations\diffusers\unet.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations\diffusers
copying deepspeed\model_implementations\diffusers\vae.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations\diffusers
copying deepspeed\model_implementations\diffusers\__init__.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations\diffusers
creating build\lib.win-amd64-3.10\deepspeed\model_implementations\transformers
copying deepspeed\model_implementations\transformers\clip_encoder.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations\transformers
copying deepspeed\model_implementations\transformers\ds_transformer.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations\transformers
copying deepspeed\model_implementations\transformers\__init__.py -> build\lib.win-amd64-3.10\deepspeed\model_implementations\transformers
creating build\lib.win-amd64-3.10\deepspeed\ops\adagrad
copying deepspeed\ops\adagrad\cpu_adagrad.py -> build\lib.win-amd64-3.10\deepspeed\ops\adagrad
copying deepspeed\ops\adagrad\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\adagrad
creating build\lib.win-amd64-3.10\deepspeed\ops\adam
copying deepspeed\ops\adam\cpu_adam.py -> build\lib.win-amd64-3.10\deepspeed\ops\adam
copying deepspeed\ops\adam\fused_adam.py -> build\lib.win-amd64-3.10\deepspeed\ops\adam
copying deepspeed\ops\adam\multi_tensor_apply.py -> build\lib.win-amd64-3.10\deepspeed\ops\adam
copying deepspeed\ops\adam\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\adam
creating build\lib.win-amd64-3.10\deepspeed\ops\aio
copying deepspeed\ops\aio\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\aio
creating build\lib.win-amd64-3.10\deepspeed\ops\lamb
copying deepspeed\ops\lamb\fused_lamb.py -> build\lib.win-amd64-3.10\deepspeed\ops\lamb
copying deepspeed\ops\lamb\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\lamb
creating build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\all_ops.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\async_io.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\builder.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\builder_names.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\cpu_adagrad.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\cpu_adam.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\fused_adam.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\fused_lamb.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\quantizer.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\sparse_attn.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\spatial_inference.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\stochastic_transformer.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\transformer.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\transformer_inference.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\utils.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
copying deepspeed\ops\op_builder\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\op_builder
creating build\lib.win-amd64-3.10\deepspeed\ops\quantizer
copying deepspeed\ops\quantizer\quantizer.py -> build\lib.win-amd64-3.10\deepspeed\ops\quantizer
copying deepspeed\ops\quantizer\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\quantizer
creating build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\bert_sparse_self_attention.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\matmul.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\softmax.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\sparse_attention_utils.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\sparse_self_attention.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\sparsity_config.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
copying deepspeed\ops\sparse_attention\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention
creating build\lib.win-amd64-3.10\deepspeed\ops\transformer
copying deepspeed\ops\transformer\transformer.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer
copying deepspeed\ops\transformer\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer
creating build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention\trsrc
copying deepspeed\ops\sparse_attention\trsrc\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention\trsrc
creating build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\bias_add.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\config.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\diffusers_2d_transformer.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\diffusers_attention.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\diffusers_transformer_block.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\ds_attention.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\ds_mlp.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\moe_inference.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\triton_ops.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
copying deepspeed\ops\transformer\inference\__init__.py -> build\lib.win-amd64-3.10\deepspeed\ops\transformer\inference
creating build\lib.win-amd64-3.10\deepspeed\profiling\flops_profiler
copying deepspeed\profiling\flops_profiler\profiler.py -> build\lib.win-amd64-3.10\deepspeed\profiling\flops_profiler
copying deepspeed\profiling\flops_profiler\__init__.py -> build\lib.win-amd64-3.10\deepspeed\profiling\flops_profiler
creating build\lib.win-amd64-3.10\deepspeed\runtime\activation_checkpointing
copying deepspeed\runtime\activation_checkpointing\checkpointing.py -> build\lib.win-amd64-3.10\deepspeed\runtime\activation_checkpointing
copying deepspeed\runtime\activation_checkpointing\config.py -> build\lib.win-amd64-3.10\deepspeed\runtime\activation_checkpointing
copying deepspeed\runtime\activation_checkpointing\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\activation_checkpointing
creating build\lib.win-amd64-3.10\deepspeed\runtime\checkpoint_engine
copying deepspeed\runtime\checkpoint_engine\checkpoint_engine.py -> build\lib.win-amd64-3.10\deepspeed\runtime\checkpoint_engine
copying deepspeed\runtime\checkpoint_engine\nebula_checkpoint_engine.py -> build\lib.win-amd64-3.10\deepspeed\runtime\checkpoint_engine
copying deepspeed\runtime\checkpoint_engine\torch_checkpoint_engine.py -> build\lib.win-amd64-3.10\deepspeed\runtime\checkpoint_engine
copying deepspeed\runtime\checkpoint_engine\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\checkpoint_engine
creating build\lib.win-amd64-3.10\deepspeed\runtime\comm
copying deepspeed\runtime\comm\coalesced_collectives.py -> build\lib.win-amd64-3.10\deepspeed\runtime\comm
copying deepspeed\runtime\comm\mpi.py -> build\lib.win-amd64-3.10\deepspeed\runtime\comm
copying deepspeed\runtime\comm\nccl.py -> build\lib.win-amd64-3.10\deepspeed\runtime\comm
copying deepspeed\runtime\comm\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\comm
creating build\lib.win-amd64-3.10\deepspeed\runtime\compression
copying deepspeed\runtime\compression\cupy.py -> build\lib.win-amd64-3.10\deepspeed\runtime\compression
copying deepspeed\runtime\compression\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\compression
creating build\lib.win-amd64-3.10\deepspeed\runtime\data_pipeline
copying deepspeed\runtime\data_pipeline\curriculum_scheduler.py -> build\lib.win-amd64-3.10\deepspeed\runtime\data_pipeline
copying deepspeed\runtime\data_pipeline\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\data_pipeline
creating build\lib.win-amd64-3.10\deepspeed\runtime\fp16
copying deepspeed\runtime\fp16\fused_optimizer.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16
copying deepspeed\runtime\fp16\loss_scaler.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16
copying deepspeed\runtime\fp16\unfused_optimizer.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16
copying deepspeed\runtime\fp16\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16
creating build\lib.win-amd64-3.10\deepspeed\runtime\pipe
copying deepspeed\runtime\pipe\engine.py -> build\lib.win-amd64-3.10\deepspeed\runtime\pipe
copying deepspeed\runtime\pipe\module.py -> build\lib.win-amd64-3.10\deepspeed\runtime\pipe
copying deepspeed\runtime\pipe\p2p.py -> build\lib.win-amd64-3.10\deepspeed\runtime\pipe
copying deepspeed\runtime\pipe\schedule.py -> build\lib.win-amd64-3.10\deepspeed\runtime\pipe
copying deepspeed\runtime\pipe\topology.py -> build\lib.win-amd64-3.10\deepspeed\runtime\pipe
copying deepspeed\runtime\pipe\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\pipe
creating build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\aio_config.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\async_swapper.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\constants.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\optimizer_utils.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\partitioned_optimizer_swapper.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\partitioned_param_swapper.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\pipelined_optimizer_swapper.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\utils.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
copying deepspeed\runtime\swap_tensor\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\swap_tensor
creating build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\config.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\contiguous_memory_allocator.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\linear.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\offload_config.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\parameter_offload.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\partitioned_param_coordinator.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\partition_parameters.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\stage3.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\stage_1_and_2.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\test.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\tiling.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\utils.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
copying deepspeed\runtime\zero\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\zero
creating build\lib.win-amd64-3.10\deepspeed\runtime\fp16\onebit
copying deepspeed\runtime\fp16\onebit\adam.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16\onebit
copying deepspeed\runtime\fp16\onebit\lamb.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16\onebit
copying deepspeed\runtime\fp16\onebit\zoadam.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16\onebit
copying deepspeed\runtime\fp16\onebit\__init__.py -> build\lib.win-amd64-3.10\deepspeed\runtime\fp16\onebit
running egg_info
writing deepspeed.egg-info\PKG-INFO
writing dependency_links to deepspeed.egg-info\dependency_links.txt
writing entry points to deepspeed.egg-info\entry_points.txt
writing requirements to deepspeed.egg-info\requires.txt
writing top-level names to deepspeed.egg-info\top_level.txt
reading manifest file 'deepspeed.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST_win.in'
warning: no previously-included files matching '*.cpp' found under directory 'deepspeed\ops\csrc'
warning: no previously-included files matching '*.h' found under directory 'deepspeed\ops\csrc'
warning: no previously-included files matching '*.cu' found under directory 'deepspeed\ops\csrc'
warning: no previously-included files matching '*.cuh' found under directory 'deepspeed\ops\csrc'
warning: no previously-included files matching '*.cc' found under directory 'deepspeed\ops\csrc'
no previously-included directories found matching 'csrc'
no previously-included directories found matching 'op_builder'
adding license file 'LICENSE'
writing manifest file 'deepspeed.egg-info\SOURCES.txt'
copying deepspeed\ops\sparse_attention\trsrc\matmul.tr -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention\trsrc
copying deepspeed\ops\sparse_attention\trsrc\softmax_bwd.tr -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention\trsrc
copying deepspeed\ops\sparse_attention\trsrc\softmax_fwd.tr -> build\lib.win-amd64-3.10\deepspeed\ops\sparse_attention\trsrc
running build_ext
C:\Users\myUsername\Anaconda3\envs\huggingface\lib\site-packages\torch\utils\cpp_extension.py:346: UserWarning: Error checking compiler version for cl: [WinError 2] The system cannot find the file specified
  warnings.warn(f'Error checking compiler version for {compiler}: {error}')
building 'deepspeed.ops.adam.cpu_adam_op' extension
creating build\temp.win-amd64-3.10
creating build\temp.win-amd64-3.10\Release
creating build\temp.win-amd64-3.10\Release\csrc
creating build\temp.win-amd64-3.10\Release\csrc\adam
creating build\temp.win-amd64-3.10\Release\csrc\common
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -Icsrc/includes "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\include" -IC:\Users\myUsername\Anaconda3\envs\huggingface\lib\site-packages\torch\include -IC:\Users\myUsername\Anaconda3\envs\huggingface\lib\site-packages\torch\include\torch\csrc\api\include -IC:\Users\myUsername\Anaconda3\envs\huggingface\lib\site-packages\torch\include\TH -IC:\Users\myUsername\Anaconda3\envs\huggingface\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\include" -IC:\Users\myUsername\Anaconda3\envs\huggingface\include -IC:\Users\myUsername\Anaconda3\envs\huggingface\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /EHsc /Tpcsrc/adam/cpu_adam.cpp /Fobuild\temp.win-amd64-3.10\Release\csrc/adam/cpu_adam.obj /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -O2 "-LC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\lib64" -lcudart -lcublas -g -march=native -fopenmp -D__AVX256__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=cpu_adam_op -D_GLIBCXX_USE_CXX11_ABI=0
cl : Command line warning D9002 : ignoring unknown option '-LC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\lib64'
cl : Command line warning D9002 : ignoring unknown option '-lcudart'
cl : Command line warning D9002 : ignoring unknown option '-lcublas'
cl : Command line warning D9002 : ignoring unknown option '-g'
cl : Command line warning D9002 : ignoring unknown option '-march=native'
cl : Command line warning D9002 : ignoring unknown option '-fopenmp'
cpu_adam.cpp
C:\Users\myUsername\Desktop\DeepSpeed\csrc\includes\quantization.h(12): error C3646: 'high': unknown override specifier
C:\Users\myUsername\Desktop\DeepSpeed\csrc\includes\quantization.h(12): error C2059: syntax error: ':'
C:\Users\myUsername\Desktop\DeepSpeed\csrc\includes\quantization.h(12): error C2334: unexpected token(s) preceding ':'; skipping apparent function body
C:\Users\myUsername\Desktop\DeepSpeed\csrc\includes\quantization.h(20): error C2059: syntax error: ''template<''
C:\Users\myUsername\Desktop\DeepSpeed\csrc\includes\quantization.h(29): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
C:\Users\myUsername\Desktop\DeepSpeed\csrc\includes\quantization.h(29): error C2143: syntax error: missing ',' before '*'
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(19): error C2039: 'int8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(19): error C2873: 'int8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(20): error C2039: 'int16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(20): error C2873: 'int16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(21): error C2039: 'int32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(21): error C2873: 'int32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(22): error C2039: 'int64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(22): error C2873: 'int64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(23): error C2039: 'uint8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(23): error C2873: 'uint8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(24): error C2039: 'uint16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(24): error C2873: 'uint16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(25): error C2039: 'uint32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(25): error C2873: 'uint32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(26): error C2039: 'uint64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(26): error C2873: 'uint64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(28): error C2039: 'int_least8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(28): error C2873: 'int_least8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(29): error C2039: 'int_least16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(29): error C2873: 'int_least16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(30): error C2039: 'int_least32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(30): error C2873: 'int_least32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(31): error C2039: 'int_least64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(31): error C2873: 'int_least64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(32): error C2039: 'uint_least8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(32): error C2873: 'uint_least8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(33): error C2039: 'uint_least16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(33): error C2873: 'uint_least16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(34): error C2039: 'uint_least32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(34): error C2873: 'uint_least32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(35): error C2039: 'uint_least64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(35): error C2873: 'uint_least64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(37): error C2039: 'int_fast8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(37): error C2873: 'int_fast8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(38): error C2039: 'int_fast16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(38): error C2873: 'int_fast16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(39): error C2039: 'int_fast32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(39): error C2873: 'int_fast32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(40): error C2039: 'int_fast64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(40): error C2873: 'int_fast64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(41): error C2039: 'uint_fast8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(41): error C2873: 'uint_fast8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(42): error C2039: 'uint_fast16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(42): error C2873: 'uint_fast16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(43): error C2039: 'uint_fast32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(43): error C2873: 'uint_fast32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(44): error C2039: 'uint_fast64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(44): error C2873: 'uint_fast64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(46): error C2039: 'intmax_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(46): error C2873: 'intmax_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(48): error C2039: 'uintmax_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(48): error C2873: 'uintmax_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(53): error C2039: 'int8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(53): error C2873: 'int8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(54): error C2039: 'int16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(54): error C2873: 'int16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(55): error C2039: 'int32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(55): error C2873: 'int32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(56): error C2039: 'int64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(56): error C2873: 'int64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(57): error C2039: 'uint8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(57): error C2873: 'uint8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(58): error C2039: 'uint16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(58): error C2873: 'uint16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(59): error C2039: 'uint32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(59): error C2873: 'uint32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(60): error C2039: 'uint64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(60): error C2873: 'uint64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(62): error C2039: 'int_least8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(62): error C2873: 'int_least8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(63): error C2039: 'int_least16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(63): error C2873: 'int_least16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(64): error C2039: 'int_least32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(64): error C2873: 'int_least32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(65): error C2039: 'int_least64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(65): error C2873: 'int_least64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(66): error C2039: 'uint_least8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(66): error C2873: 'uint_least8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(67): error C2039: 'uint_least16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(67): error C2873: 'uint_least16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(68): error C2039: 'uint_least32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(68): error C2873: 'uint_least32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(69): error C2039: 'uint_least64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(69): error C2873: 'uint_least64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(71): error C2039: 'int_fast8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(71): error C2873: 'int_fast8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(72): error C2039: 'int_fast16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(72): error C2873: 'int_fast16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(73): error C2039: 'int_fast32_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(73): error C2873: 'int_fast32_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(74): error C2039: 'int_fast64_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(74): error C2873: 'int_fast64_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(75): error C2039: 'uint_fast8_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(75): error C2873: 'uint_fast8_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(76): error C2039: 'uint_fast16_t': is not a member of '`global namespace''
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(76): error C2873: 'uint_fast16_t': symbol cannot be used in a using-declaration
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.20.27508\include\cstdint(76): fatal error C1003: error count exceeds 100; stopping compilation
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.20.27508\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
Eichhof commented 1 year ago

Any help on this? I could not find a solution.

genbs commented 1 year ago

Same problem

Eichhof commented 1 year ago

Was somebody able to get Deepseed running for inference on Windows 10?

marcoseduardopm commented 1 year ago

(Note: these steps are for the interference only mode) After trying forever, I got it working. That's what I have done:

To install the generated .whl, just use: For Python 3.10 version: pip install deepspeed-0.8.3+6eca037c-cp310-cp310-win_amd64.whl For Pytohn 3.9 version: pip install deepspeed-0.8.3+4d27225f-cp39-cp39-win_amd64.whl

Extra Notes: Note: Tytorch version 1.13.1 with CUDA 11.7 also worked for me, but since it is an older version, I did not mention it in the steps above. If you need that version, install using "conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia"

About the replacement of file pt_binding.cpp: all I did was change lines 531, 532, 539, and 540: New Lines 531 and 532: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()),

New lines 539 and 540: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()),

For anyone that just want the final .whl to install using python, here it is (no prayers needed): https://drive.google.com/drive/folders/117GSNHcJyzvMPTftl0aPBSwQVsU-z4bM?usp=sharing

LeXwDeX commented 1 year ago

(Note: these steps are for the interference only mode) After trying forever, I got it working. That's what I have done:

  • Install the vs build tool 2019. If you already have it installed, repair it;
  • Install Miniconda (if you haven't it already);
  • Install CUDA 11.7 from https://developer.nvidia.com/cuda-11-7-0-download-archive ;
  • Open "Anaconda Prompt (MiniConda3)";
  • Create a python 3.10 env using: "conda create -n dsenv python=3.10.6"
  • Activate the conda env using "conda activate dsenv";
  • Install Pytorch and CUDA using: "conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia";
  • Close anaconda prompt;
  • Open the Start -> "x64 Native Tools Command Prompt for VS 2019";
  • Initialize conda on the Command prompt using "conda init cmd.exe";
  • Reopen the "x64 Native Tools Command Prompt for VS 2019" AS AN ADMINISTRATOR;
  • Activate the conda env using "conda activate dsenv";
  • Go to your root folder (could be c:\ or any other) and clone que DeepSpeed project "git clone https://github.com/microsoft/DeepSpeed";
  • Depending on the fixes of the DeepSpeed repository, this step might or not be needed: Download here this file (https://drive.google.com/drive/folders/11EYHosWfDLrrVbniBLV1j82qeurpGlvX?usp=sharing) and replace the file at DeepSpeed\csrc\transformer\inference\csrc\pt_binding.cpp (see comments below);
  • Go to the deepspeed folder using "cd DeepSpeed";
  • Make 10 prayers to your god and try to install using "build_win.bat";
  • A .whl will be created in the dist folder.

To install the generated .whl, just use: For Python 3.10 version: pip install deepspeed-0.8.3+6eca037c-cp310-cp310-win_amd64.whl For Pytohn 3.9 version: pip install deepspeed-0.8.3+4d27225f-cp39-cp39-win_amd64.whl

Extra Notes: Note: Tytorch version 1.13.1 with CUDA 11.7 also worked for me, but since it is an older version, I did not mention it in the steps above. If you need that version, install using "conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia"

About the replacement of file pt_binding.cpp: all I did was change lines 531, 532, 539, and 540: New Lines 531 and 532: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()),

New lines 539 and 540: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()),

For anyone that just want the final .whl to install using python, here it is (no prayers needed): https://drive.google.com/drive/folders/117GSNHcJyzvMPTftl0aPBSwQVsU-z4bM?usp=sharing

aslo report "LINK : fatal error LNK1181: cannot open input file 'aio.lib'" but use ur whl file as well ,thanks!

marcoseduardopm commented 1 year ago

(Note: these steps are for the interference only mode) After trying forever, I got it working. That's what I have done:

  • Install the vs build tool 2019. If you already have it installed, repair it;
  • Install Miniconda (if you haven't it already);
  • Install CUDA 11.7 from https://developer.nvidia.com/cuda-11-7-0-download-archive ;
  • Open "Anaconda Prompt (MiniConda3)";
  • Create a python 3.10 env using: "conda create -n dsenv python=3.10.6"
  • Activate the conda env using "conda activate dsenv";
  • Install Pytorch and CUDA using: "conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia";
  • Close anaconda prompt;
  • Open the Start -> "x64 Native Tools Command Prompt for VS 2019";
  • Initialize conda on the Command prompt using "conda init cmd.exe";
  • Reopen the "x64 Native Tools Command Prompt for VS 2019" AS AN ADMINISTRATOR;
  • Activate the conda env using "conda activate dsenv";
  • Go to your root folder (could be c:\ or any other) and clone que DeepSpeed project "git clone https://github.com/microsoft/DeepSpeed";
  • Depending on the fixes of the DeepSpeed repository, this step might or not be needed: Download here this file (https://drive.google.com/drive/folders/11EYHosWfDLrrVbniBLV1j82qeurpGlvX?usp=sharing) and replace the file at DeepSpeed\csrc\transformer\inference\csrc\pt_binding.cpp (see comments below);
  • Go to the deepspeed folder using "cd DeepSpeed";
  • Make 10 prayers to your god and try to install using "build_win.bat";
  • A .whl will be created in the dist folder.

To install the generated .whl, just use: For Python 3.10 version: pip install deepspeed-0.8.3+6eca037c-cp310-cp310-win_amd64.whl For Pytohn 3.9 version: pip install deepspeed-0.8.3+4d27225f-cp39-cp39-win_amd64.whl Extra Notes: Note: Tytorch version 1.13.1 with CUDA 11.7 also worked for me, but since it is an older version, I did not mention it in the steps above. If you need that version, install using "conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia" About the replacement of file pt_binding.cpp: all I did was change lines 531, 532, 539, and 540: New Lines 531 and 532: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()), New lines 539 and 540: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()), For anyone that just want the final .whl to install using python, here it is (no prayers needed): https://drive.google.com/drive/folders/117GSNHcJyzvMPTftl0aPBSwQVsU-z4bM?usp=sharing

aslo report "LINK : fatal error LNK1181: cannot open input file 'aio.lib'" but use ur whl file as well ,thanks!

Did you use build_win.bat to build so that DS_BUILD_AIO is set to 0?

AngelTs commented 1 year ago

(Note: these steps are for the interference only mode) After trying forever, I got it working. That's what I have done:

... About the replacement of file pt_binding.cpp: all I did was change lines 531, 532, 539, and 540: New Lines 531 and 532: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()),

New lines 539 and 540: {static_cast(hidden_dim Context::Instance().GetMaxTokenLenght()), static_cast(k Context::Instance().GetMaxTokenLenght()),

...

These sentences have at least four errors- Not "Context::Instance()", but "InferenceContext::Instance()" and still can't compile, because must use <> after static_cast: error C2760: syntax error: unexpected token '(', expected '<'

The correct four lines to compile without a problem are: {static_cast(hidden_dim InferenceContext::Instance().GetMaxTokenLenght()), static_cast(k InferenceContext::Instance().GetMaxTokenLenght()), {static_cast(hidden_dim InferenceContext::Instance().GetMaxTokenLenght()), static_cast(k InferenceContext::Instance().GetMaxTokenLenght()),

loadams commented 2 months ago

This should be resolved with our latest Windows whl - please check the blog post and open a new issue if you are still hitting this.