error

[cxq1]

pyTorch version: 1.9.0+cu111 Is debug build: False CUDA used to build PyTorch: 11.1

D-WSCL/wetectron/csrc/cuda/ROIPool_cuda.cu line=151 error=209 : no kernel image is available for execution on the device

[jinhseo]

Such error message could be triggered if you didn't install apex properly. Please follow the environment setup I provided. It may different depending on your pytorch version and GPU capability, and even the pytorch version in your error message doesn't match what I suggested.

[cxq1]

Such error message could be triggered if you didn't install apex properly. Please follow the environment setup I provided. It may different depending on your pytorch version and GPU capability, and even the pytorch version in your error message doesn't match what I suggested.

PyTorch version: 1.7.1+cu110 Is debug build: False CUDA used to build PyTorch: 11.0 Python version: 3.7 (64-bit runtime) Is CUDA available: True

The problem remains

[jinhseo]

Do you want to share your log when you installing apex? or check you got any error when you run: pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./ and python setup.py build develop

[cxq1]

I use the code "pip install -v --no-cache-dir ./" can successfully installed

sing pip 22.2.1 from /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/pip (python 3.7) Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Processing /home/cxq/dp_work/objdetection/OD-WSCL/apex Running command python setup.py egg_info

torch.version = 1.7.1+cu110

running egg_info creating /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info writing /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/PKG-INFO writing dependency_links to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/dependency_links.txt writing requirements to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/requires.txt writing top-level names to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/top_level.txt writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' Preparing metadata (setup.py) ... done Requirement already satisfied: packaging>20.6 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from apex==0.1) (21.3) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from packaging>20.6->apex==0.1) (3.0.9) Building wheels for collected packages: apex Running command python setup.py bdist_wheel

torch.version = 1.7.1+cu110

running bdist_wheel running build running build_py installing to build/bdist.linux-x86_64/wheel running install running install_lib creating build/bdist.linux-x86_64/wheel creating build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/compat.py -> build/bdist.linux-x86_64/wheel/apex/amp creating build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/init.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/tensor_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/torch_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/functional_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/rnn_compat.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/wrap.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_initialize.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/scaler.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_process_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/opt.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/amp.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_amp_state.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/version.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/utils.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/handle.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/frontend.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/_autocast_utils.py -> build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/init.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/fp16util.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/loss_scaler.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/init.py -> build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/normalization copying build/lib/apex/normalization/init.py -> build/bdist.linux-x86_64/wheel/apex/normalization copying build/lib/apex/normalization/fused_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/normalization creating build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/cells.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/init.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/RNNBackend.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/models.py -> build/bdist.linux-x86_64/wheel/apex/RNN creating build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_adagrad.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_novograd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_mixed_precision_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/optimizers creating build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/optimized_sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/distributed.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/multiproc.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/optimized_sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/LARC.py -> build/bdist.linux-x86_64/wheel/apex/parallel creating build/bdist.linux-x86_64/wheel/apex/contrib creating build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/transducer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/_transducer_ref.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer creating build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/xentropy/softmax_xentropy.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib creating build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad copying build/lib/apex/contrib/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad copying build/lib/apex/contrib/clip_grad/clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad creating build/bdist.linux-x86_64/wheel/apex/contrib/test creating build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/test_transducer_joint.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/test_transducer_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer creating build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/xentropy/test_label_smoothing.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test creating build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad copying build/lib/apex/contrib/test/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad copying build/lib/apex/contrib/test/clip_grad/test_clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad creating build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/test_dist_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/test_distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers creating build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_mha_fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn creating build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d copying build/lib/apex/contrib/test/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d copying build/lib/apex/contrib/test/index_mul_2d/test_index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d creating build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu copying build/lib/apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu copying build/lib/apex/contrib/test/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu creating build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck copying build/lib/apex/contrib/test/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck copying build/lib/apex/contrib/test/bottleneck/test_bottleneck_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck creating build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn copying build/lib/apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn copying build/lib/apex/contrib/test/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn creating build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm copying build/lib/apex/contrib/test/layer_norm/test_fast_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm copying build/lib/apex/contrib/test/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm creating build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha copying build/lib/apex/contrib/test/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha copying build/lib/apex/contrib/test/fmha/test_fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha creating build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory copying build/lib/apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory copying build/lib/apex/contrib/test/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory creating build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss copying build/lib/apex/contrib/test/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss copying build/lib/apex/contrib/test/focal_loss/test_focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss creating build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/distributed_fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers creating build/bdist.linux-x86_64/wheel/apex/contrib/groupbn copying build/lib/apex/contrib/groupbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn copying build/lib/apex/contrib/groupbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn creating build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn creating build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d copying build/lib/apex/contrib/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d copying build/lib/apex/contrib/index_mul_2d/index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/permutation_lib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/sparse_masklib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/asp.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/channel_swap.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels creating build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu copying build/lib/apex/contrib/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu copying build/lib/apex/contrib/conv_bias_relu/conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu creating build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/bottleneck.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/test.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/halo_exchangers.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck creating build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn copying build/lib/apex/contrib/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn copying build/lib/apex/contrib/cudnn_gbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn creating build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm copying build/lib/apex/contrib/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm copying build/lib/apex/contrib/layer_norm/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm creating build/bdist.linux-x86_64/wheel/apex/contrib/fmha copying build/lib/apex/contrib/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha copying build/lib/apex/contrib/fmha/fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha creating build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/peer_halo_exchanger_1d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/peer_memory.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory creating build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss copying build/lib/apex/contrib/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss copying build/lib/apex/contrib/focal_loss/focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss creating build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply copying build/lib/apex/multi_tensor_apply/init.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply copying build/lib/apex/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply creating build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/data.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/random.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/cross_entropy.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/mappings.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/layers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/memory.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel creating build/bdist.linux-x86_64/wheel/apex/transformer/amp copying build/lib/apex/transformer/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp copying build/lib/apex/transformer/amp/grad_scaler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp creating build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/commons.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/global_vars.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_gpt.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_bert.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_transformer_lm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/arguments.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/distributed_test_base.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing creating build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_data/_batchsampler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_data/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_ucc_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/log_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/common.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/_timers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/p2p_communication.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/microbatches.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/enums.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/functional/fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/functional/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/layers/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/layers/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/parallel_state.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/fused_dense copying build/lib/apex/fused_dense/fused_dense.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense copying build/lib/apex/fused_dense/init.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense creating build/bdist.linux-x86_64/wheel/apex/mlp copying build/lib/apex/mlp/mlp.py -> build/bdist.linux-x86_64/wheel/apex/mlp copying build/lib/apex/mlp/init.py -> build/bdist.linux-x86_64/wheel/apex/mlp running install_egg_info running egg_info writing apex.egg-info/PKG-INFO writing dependency_links to apex.egg-info/dependency_links.txt writing requirements to apex.egg-info/requires.txt writing top-level names to apex.egg-info/top_level.txt reading manifest file 'apex.egg-info/SOURCES.txt' writing manifest file 'apex.egg-info/SOURCES.txt' Copying apex.egg-info to build/bdist.linux-x86_64/wheel/apex-0.1-py3.7.egg-info running install_scripts adding license file "LICENSE" (matched pattern "LICEN[CS]E*") creating build/bdist.linux-x86_64/wheel/apex-0.1.dist-info/WHEEL creating '/tmp/pip-wheel-o4d_cpbt/apex-0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it adding 'apex/init.py' adding 'apex/_autocast_utils.py' adding 'apex/RNN/RNNBackend.py' adding 'apex/RNN/init.py' adding 'apex/RNN/cells.py' adding 'apex/RNN/models.py' adding 'apex/amp/init.py' adding 'apex/amp/version.py' adding 'apex/amp/_amp_state.py' adding 'apex/amp/_initialize.py' adding 'apex/amp/_process_optimizer.py' adding 'apex/amp/amp.py' adding 'apex/amp/compat.py' adding 'apex/amp/frontend.py' adding 'apex/amp/handle.py' adding 'apex/amp/opt.py' adding 'apex/amp/rnn_compat.py' adding 'apex/amp/scaler.py' adding 'apex/amp/utils.py' adding 'apex/amp/wrap.py' adding 'apex/amp/lists/init.py' adding 'apex/amp/lists/functional_overrides.py' adding 'apex/amp/lists/tensor_overrides.py' adding 'apex/amp/lists/torch_overrides.py' adding 'apex/contrib/init.py' adding 'apex/contrib/bottleneck/init.py' adding 'apex/contrib/bottleneck/bottleneck.py' adding 'apex/contrib/bottleneck/halo_exchangers.py' adding 'apex/contrib/bottleneck/test.py' adding 'apex/contrib/clip_grad/init.py' adding 'apex/contrib/clip_grad/clip_grad.py' adding 'apex/contrib/conv_bias_relu/init.py' adding 'apex/contrib/conv_bias_relu/conv_bias_relu.py' adding 'apex/contrib/cudnn_gbn/init.py' adding 'apex/contrib/cudnn_gbn/batch_norm.py' adding 'apex/contrib/fmha/init.py' adding 'apex/contrib/fmha/fmha.py' adding 'apex/contrib/focal_loss/init.py' adding 'apex/contrib/focal_loss/focal_loss.py' adding 'apex/contrib/groupbn/init.py' adding 'apex/contrib/groupbn/batch_norm.py' adding 'apex/contrib/index_mul_2d/init.py' adding 'apex/contrib/index_mul_2d/index_mul_2d.py' adding 'apex/contrib/layer_norm/init.py' adding 'apex/contrib/layer_norm/layer_norm.py' adding 'apex/contrib/multihead_attn/init.py' adding 'apex/contrib/multihead_attn/encdec_multihead_attn.py' adding 'apex/contrib/multihead_attn/encdec_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py' adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py' adding 'apex/contrib/multihead_attn/mask_softmax_dropout_func.py' adding 'apex/contrib/multihead_attn/self_multihead_attn.py' adding 'apex/contrib/multihead_attn/self_multihead_attn_func.py' adding 'apex/contrib/optimizers/init.py' adding 'apex/contrib/optimizers/distributed_fused_adam.py' adding 'apex/contrib/optimizers/distributed_fused_lamb.py' adding 'apex/contrib/optimizers/fp16_optimizer.py' adding 'apex/contrib/optimizers/fused_adam.py' adding 'apex/contrib/optimizers/fused_lamb.py' adding 'apex/contrib/optimizers/fused_sgd.py' adding 'apex/contrib/peer_memory/init.py' adding 'apex/contrib/peer_memory/peer_halo_exchanger_1d.py' adding 'apex/contrib/peer_memory/peer_memory.py' adding 'apex/contrib/sparsity/init.py' adding 'apex/contrib/sparsity/asp.py' adding 'apex/contrib/sparsity/permutation_lib.py' adding 'apex/contrib/sparsity/sparse_masklib.py' adding 'apex/contrib/sparsity/permutation_search_kernels/init.py' adding 'apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py' adding 'apex/contrib/sparsity/permutation_search_kernels/channel_swap.py' adding 'apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py' adding 'apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py' adding 'apex/contrib/test/init.py' adding 'apex/contrib/test/bottleneck/init.py' adding 'apex/contrib/test/bottleneck/test_bottleneck_module.py' adding 'apex/contrib/test/clip_grad/init.py' adding 'apex/contrib/test/clip_grad/test_clip_grad.py' adding 'apex/contrib/test/conv_bias_relu/init.py' adding 'apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py' adding 'apex/contrib/test/cudnn_gbn/init.py' adding 'apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py' adding 'apex/contrib/test/fmha/init.py' adding 'apex/contrib/test/fmha/test_fmha.py' adding 'apex/contrib/test/focal_loss/init.py' adding 'apex/contrib/test/focal_loss/test_focal_loss.py' adding 'apex/contrib/test/index_mul_2d/init.py' adding 'apex/contrib/test/index_mul_2d/test_index_mul_2d.py' adding 'apex/contrib/test/layer_norm/init.py' adding 'apex/contrib/test/layer_norm/test_fast_layer_norm.py' adding 'apex/contrib/test/multihead_attn/init.py' adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py' adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py' adding 'apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py' adding 'apex/contrib/test/multihead_attn/test_mha_fused_softmax.py' adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn.py' adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py' adding 'apex/contrib/test/optimizers/init.py' adding 'apex/contrib/test/optimizers/test_dist_adam.py' adding 'apex/contrib/test/optimizers/test_distributed_fused_lamb.py' adding 'apex/contrib/test/peer_memory/init.py' adding 'apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py' adding 'apex/contrib/test/transducer/init.py' adding 'apex/contrib/test/transducer/test_transducer_joint.py' adding 'apex/contrib/test/transducer/test_transducer_loss.py' adding 'apex/contrib/test/xentropy/init.py' adding 'apex/contrib/test/xentropy/test_label_smoothing.py' adding 'apex/contrib/transducer/init.py' adding 'apex/contrib/transducer/_transducer_ref.py' adding 'apex/contrib/transducer/transducer.py' adding 'apex/contrib/xentropy/init.py' adding 'apex/contrib/xentropy/softmax_xentropy.py' adding 'apex/fp16_utils/init.py' adding 'apex/fp16_utils/fp16_optimizer.py' adding 'apex/fp16_utils/fp16util.py' adding 'apex/fp16_utils/loss_scaler.py' adding 'apex/fused_dense/init.py' adding 'apex/fused_dense/fused_dense.py' adding 'apex/mlp/init.py' adding 'apex/mlp/mlp.py' adding 'apex/multi_tensor_apply/init.py' adding 'apex/multi_tensor_apply/multi_tensor_apply.py' adding 'apex/normalization/init.py' adding 'apex/normalization/fused_layer_norm.py' adding 'apex/optimizers/init.py' adding 'apex/optimizers/fused_adagrad.py' adding 'apex/optimizers/fused_adam.py' adding 'apex/optimizers/fused_lamb.py' adding 'apex/optimizers/fused_mixed_precision_lamb.py' adding 'apex/optimizers/fused_novograd.py' adding 'apex/optimizers/fused_sgd.py' adding 'apex/parallel/LARC.py' adding 'apex/parallel/init.py' adding 'apex/parallel/distributed.py' adding 'apex/parallel/multiproc.py' adding 'apex/parallel/optimized_sync_batchnorm.py' adding 'apex/parallel/optimized_sync_batchnorm_kernel.py' adding 'apex/parallel/sync_batchnorm.py' adding 'apex/parallel/sync_batchnorm_kernel.py' adding 'apex/transformer/init.py' adding 'apex/transformer/_ucc_util.py' adding 'apex/transformer/enums.py' adding 'apex/transformer/log_util.py' adding 'apex/transformer/microbatches.py' adding 'apex/transformer/parallel_state.py' adding 'apex/transformer/utils.py' adding 'apex/transformer/_data/init.py' adding 'apex/transformer/_data/_batchsampler.py' adding 'apex/transformer/amp/init.py' adding 'apex/transformer/amp/grad_scaler.py' adding 'apex/transformer/functional/init.py' adding 'apex/transformer/functional/fused_softmax.py' adding 'apex/transformer/layers/init.py' adding 'apex/transformer/layers/layer_norm.py' adding 'apex/transformer/pipeline_parallel/init.py' adding 'apex/transformer/pipeline_parallel/_timers.py' adding 'apex/transformer/pipeline_parallel/p2p_communication.py' adding 'apex/transformer/pipeline_parallel/utils.py' adding 'apex/transformer/pipeline_parallel/schedules/init.py' adding 'apex/transformer/pipeline_parallel/schedules/common.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py' adding 'apex/transformer/tensor_parallel/init.py' adding 'apex/transformer/tensor_parallel/cross_entropy.py' adding 'apex/transformer/tensor_parallel/data.py' adding 'apex/transformer/tensor_parallel/layers.py' adding 'apex/transformer/tensor_parallel/mappings.py' adding 'apex/transformer/tensor_parallel/memory.py' adding 'apex/transformer/tensor_parallel/random.py' adding 'apex/transformer/tensor_parallel/utils.py' adding 'apex/transformer/testing/init.py' adding 'apex/transformer/testing/arguments.py' adding 'apex/transformer/testing/commons.py' adding 'apex/transformer/testing/distributed_test_base.py' adding 'apex/transformer/testing/global_vars.py' adding 'apex/transformer/testing/standalone_bert.py' adding 'apex/transformer/testing/standalone_gpt.py' adding 'apex/transformer/testing/standalone_transformer_lm.py' adding 'apex-0.1.dist-info/LICENSE' adding 'apex-0.1.dist-info/METADATA' adding 'apex-0.1.dist-info/WHEEL' adding 'apex-0.1.dist-info/top_level.txt' adding 'apex-0.1.dist-info/RECORD' removing build/bdist.linux-x86_64/wheel Building wheel for apex (setup.py) ... done Created wheel for apex: filename=apex-0.1-py3-none-any.whl size=346778 sha256=bac9bdfa231d1f15122a9c3f292beee94df5bcf1c979b3240091d5c90b3f269c Stored in directory: /tmp/pip-ephem-wheel-cache-6m7s92_k/wheels/0a/df/3d/18f4c8df382be1e9dc0b4f5c16593b2d47107867ca80e7a6f6 Successfully built apex Installing collected packages: apex Attempting uninstall: apex Found existing installation: apex 0.1 Uninstalling apex-0.1: Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex-0.1.dist-info/ Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex/ Successfully uninstalled apex-0.1 Successfully installed apex-0.1

[cxq1]

Do you want to share your log when you installing apex? or check you got any error when you run: pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./ and python setup.py build develop

AttributeError: module 'torch.distributed' has no attribute '_all_gather_base'

[ShirleySyh]

I use the code "pip install -v --no-cache-dir ./" can successfully installed

sing pip 22.2.1 from /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/pip (python 3.7) Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple Processing /home/cxq/dp_work/objdetection/OD-WSCL/apex Running command python setup.py egg_info

torch.version = 1.7.1+cu110

running egg_info creating /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info writing /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/PKG-INFO writing dependency_links to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/dependency_links.txt writing requirements to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/requires.txt writing top-level names to /tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/top_level.txt writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' writing manifest file '/tmp/pip-pip-egg-info-ku2tb53y/apex.egg-info/SOURCES.txt' Preparing metadata (setup.py) ... done Requirement already satisfied: packaging>20.6 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from apex==0.1) (21.3) Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages (from packaging>20.6->apex==0.1) (3.0.9) Building wheels for collected packages: apex Running command python setup.py bdist_wheel

torch.version = 1.7.1+cu110

running bdist_wheel running build running build_py installing to build/bdist.linux-x86_64/wheel running install running install_lib creating build/bdist.linux-x86_64/wheel creating build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/compat.py -> build/bdist.linux-x86_64/wheel/apex/amp creating build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/init.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/tensor_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/torch_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/lists/functional_overrides.py -> build/bdist.linux-x86_64/wheel/apex/amp/lists copying build/lib/apex/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/rnn_compat.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/wrap.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_initialize.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/scaler.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_process_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/opt.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/amp.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/_amp_state.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/version.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/utils.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/handle.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/amp/frontend.py -> build/bdist.linux-x86_64/wheel/apex/amp copying build/lib/apex/_autocast_utils.py -> build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/init.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/fp16util.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/loss_scaler.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/fp16_utils/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/fp16_utils copying build/lib/apex/init.py -> build/bdist.linux-x86_64/wheel/apex creating build/bdist.linux-x86_64/wheel/apex/normalization copying build/lib/apex/normalization/init.py -> build/bdist.linux-x86_64/wheel/apex/normalization copying build/lib/apex/normalization/fused_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/normalization creating build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/cells.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/init.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/RNNBackend.py -> build/bdist.linux-x86_64/wheel/apex/RNN copying build/lib/apex/RNN/models.py -> build/bdist.linux-x86_64/wheel/apex/RNN creating build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_adagrad.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_novograd.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_mixed_precision_lamb.py -> build/bdist.linux-x86_64/wheel/apex/optimizers copying build/lib/apex/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/optimizers creating build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/optimized_sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/sync_batchnorm.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/distributed.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/multiproc.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/optimized_sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/sync_batchnorm_kernel.py -> build/bdist.linux-x86_64/wheel/apex/parallel copying build/lib/apex/parallel/LARC.py -> build/bdist.linux-x86_64/wheel/apex/parallel creating build/bdist.linux-x86_64/wheel/apex/contrib creating build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/transducer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer copying build/lib/apex/contrib/transducer/_transducer_ref.py -> build/bdist.linux-x86_64/wheel/apex/contrib/transducer creating build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/xentropy/softmax_xentropy.py -> build/bdist.linux-x86_64/wheel/apex/contrib/xentropy copying build/lib/apex/contrib/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib creating build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad copying build/lib/apex/contrib/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad copying build/lib/apex/contrib/clip_grad/clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/clip_grad creating build/bdist.linux-x86_64/wheel/apex/contrib/test creating build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/test_transducer_joint.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer copying build/lib/apex/contrib/test/transducer/test_transducer_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/transducer creating build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/xentropy/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/xentropy/test_label_smoothing.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/xentropy copying build/lib/apex/contrib/test/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test creating build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad copying build/lib/apex/contrib/test/clip_grad/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad copying build/lib/apex/contrib/test/clip_grad/test_clip_grad.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/clip_grad creating build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/test_dist_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers copying build/lib/apex/contrib/test/optimizers/test_distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/optimizers creating build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_mha_fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn copying build/lib/apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/multihead_attn creating build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d copying build/lib/apex/contrib/test/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d copying build/lib/apex/contrib/test/index_mul_2d/test_index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/index_mul_2d creating build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu copying build/lib/apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu copying build/lib/apex/contrib/test/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/conv_bias_relu creating build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck copying build/lib/apex/contrib/test/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck copying build/lib/apex/contrib/test/bottleneck/test_bottleneck_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/bottleneck creating build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn copying build/lib/apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn copying build/lib/apex/contrib/test/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/cudnn_gbn creating build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm copying build/lib/apex/contrib/test/layer_norm/test_fast_layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm copying build/lib/apex/contrib/test/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/layer_norm creating build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha copying build/lib/apex/contrib/test/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha copying build/lib/apex/contrib/test/fmha/test_fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/fmha creating build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory copying build/lib/apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory copying build/lib/apex/contrib/test/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/peer_memory creating build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss copying build/lib/apex/contrib/test/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss copying build/lib/apex/contrib/test/focal_loss/test_focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/test/focal_loss creating build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/distributed_fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/distributed_fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fp16_optimizer.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_lamb.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_sgd.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers copying build/lib/apex/contrib/optimizers/fused_adam.py -> build/bdist.linux-x86_64/wheel/apex/contrib/optimizers creating build/bdist.linux-x86_64/wheel/apex/contrib/groupbn copying build/lib/apex/contrib/groupbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn copying build/lib/apex/contrib/groupbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/groupbn creating build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/self_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn copying build/lib/apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/bdist.linux-x86_64/wheel/apex/contrib/multihead_attn creating build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d copying build/lib/apex/contrib/index_mul_2d/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d copying build/lib/apex/contrib/index_mul_2d/index_mul_2d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/index_mul_2d creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/permutation_lib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/sparse_masklib.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity copying build/lib/apex/contrib/sparsity/asp.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity creating build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/channel_swap.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels copying build/lib/apex/contrib/sparsity/permutation_search_kernels/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/sparsity/permutation_search_kernels creating build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu copying build/lib/apex/contrib/conv_bias_relu/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu copying build/lib/apex/contrib/conv_bias_relu/conv_bias_relu.py -> build/bdist.linux-x86_64/wheel/apex/contrib/conv_bias_relu creating build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/bottleneck.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/test.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck copying build/lib/apex/contrib/bottleneck/halo_exchangers.py -> build/bdist.linux-x86_64/wheel/apex/contrib/bottleneck creating build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn copying build/lib/apex/contrib/cudnn_gbn/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn copying build/lib/apex/contrib/cudnn_gbn/batch_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/cudnn_gbn creating build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm copying build/lib/apex/contrib/layer_norm/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm copying build/lib/apex/contrib/layer_norm/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/contrib/layer_norm creating build/bdist.linux-x86_64/wheel/apex/contrib/fmha copying build/lib/apex/contrib/fmha/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha copying build/lib/apex/contrib/fmha/fmha.py -> build/bdist.linux-x86_64/wheel/apex/contrib/fmha creating build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/peer_halo_exchanger_1d.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory copying build/lib/apex/contrib/peer_memory/peer_memory.py -> build/bdist.linux-x86_64/wheel/apex/contrib/peer_memory creating build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss copying build/lib/apex/contrib/focal_loss/init.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss copying build/lib/apex/contrib/focal_loss/focal_loss.py -> build/bdist.linux-x86_64/wheel/apex/contrib/focal_loss creating build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply copying build/lib/apex/multi_tensor_apply/init.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply copying build/lib/apex/multi_tensor_apply/multi_tensor_apply.py -> build/bdist.linux-x86_64/wheel/apex/multi_tensor_apply creating build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/data.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/random.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/cross_entropy.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/mappings.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/layers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/memory.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel copying build/lib/apex/transformer/tensor_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/tensor_parallel creating build/bdist.linux-x86_64/wheel/apex/transformer/amp copying build/lib/apex/transformer/amp/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp copying build/lib/apex/transformer/amp/grad_scaler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/amp creating build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/commons.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/global_vars.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_gpt.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_bert.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/standalone_transformer_lm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/arguments.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing copying build/lib/apex/transformer/testing/distributed_test_base.py -> build/bdist.linux-x86_64/wheel/apex/transformer/testing creating build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_data/_batchsampler.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_data/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/_data copying build/lib/apex/transformer/_ucc_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/log_util.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel creating build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/schedules/common.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel/schedules copying build/lib/apex/transformer/pipeline_parallel/_timers.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/p2p_communication.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/pipeline_parallel/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer/pipeline_parallel copying build/lib/apex/transformer/microbatches.py -> build/bdist.linux-x86_64/wheel/apex/transformer copying build/lib/apex/transformer/enums.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/functional/fused_softmax.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/functional/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/functional copying build/lib/apex/transformer/utils.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/layers/init.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/layers/layer_norm.py -> build/bdist.linux-x86_64/wheel/apex/transformer/layers copying build/lib/apex/transformer/parallel_state.py -> build/bdist.linux-x86_64/wheel/apex/transformer creating build/bdist.linux-x86_64/wheel/apex/fused_dense copying build/lib/apex/fused_dense/fused_dense.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense copying build/lib/apex/fused_dense/init.py -> build/bdist.linux-x86_64/wheel/apex/fused_dense creating build/bdist.linux-x86_64/wheel/apex/mlp copying build/lib/apex/mlp/mlp.py -> build/bdist.linux-x86_64/wheel/apex/mlp copying build/lib/apex/mlp/init.py -> build/bdist.linux-x86_64/wheel/apex/mlp running install_egg_info running egg_info writing apex.egg-info/PKG-INFO writing dependency_links to apex.egg-info/dependency_links.txt writing requirements to apex.egg-info/requires.txt writing top-level names to apex.egg-info/top_level.txt reading manifest file 'apex.egg-info/SOURCES.txt' writing manifest file 'apex.egg-info/SOURCES.txt' Copying apex.egg-info to build/bdist.linux-x86_64/wheel/apex-0.1-py3.7.egg-info running install_scripts adding license file "LICENSE" (matched pattern "LICEN[CS]E*") creating build/bdist.linux-x86_64/wheel/apex-0.1.dist-info/WHEEL creating '/tmp/pip-wheel-o4d_cpbt/apex-0.1-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it adding 'apex/init.py' adding 'apex/_autocast_utils.py' adding 'apex/RNN/RNNBackend.py' adding 'apex/RNN/init.py' adding 'apex/RNN/cells.py' adding 'apex/RNN/models.py' adding 'apex/amp/init.py' adding 'apex/amp/version.py' adding 'apex/amp/_amp_state.py' adding 'apex/amp/_initialize.py' adding 'apex/amp/_process_optimizer.py' adding 'apex/amp/amp.py' adding 'apex/amp/compat.py' adding 'apex/amp/frontend.py' adding 'apex/amp/handle.py' adding 'apex/amp/opt.py' adding 'apex/amp/rnn_compat.py' adding 'apex/amp/scaler.py' adding 'apex/amp/utils.py' adding 'apex/amp/wrap.py' adding 'apex/amp/lists/init.py' adding 'apex/amp/lists/functional_overrides.py' adding 'apex/amp/lists/tensor_overrides.py' adding 'apex/amp/lists/torch_overrides.py' adding 'apex/contrib/init.py' adding 'apex/contrib/bottleneck/init.py' adding 'apex/contrib/bottleneck/bottleneck.py' adding 'apex/contrib/bottleneck/halo_exchangers.py' adding 'apex/contrib/bottleneck/test.py' adding 'apex/contrib/clip_grad/init.py' adding 'apex/contrib/clip_grad/clip_grad.py' adding 'apex/contrib/conv_bias_relu/init.py' adding 'apex/contrib/conv_bias_relu/conv_bias_relu.py' adding 'apex/contrib/cudnn_gbn/init.py' adding 'apex/contrib/cudnn_gbn/batch_norm.py' adding 'apex/contrib/fmha/init.py' adding 'apex/contrib/fmha/fmha.py' adding 'apex/contrib/focal_loss/init.py' adding 'apex/contrib/focal_loss/focal_loss.py' adding 'apex/contrib/groupbn/init.py' adding 'apex/contrib/groupbn/batch_norm.py' adding 'apex/contrib/index_mul_2d/init.py' adding 'apex/contrib/index_mul_2d/index_mul_2d.py' adding 'apex/contrib/layer_norm/init.py' adding 'apex/contrib/layer_norm/layer_norm.py' adding 'apex/contrib/multihead_attn/init.py' adding 'apex/contrib/multihead_attn/encdec_multihead_attn.py' adding 'apex/contrib/multihead_attn/encdec_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py' adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_func.py' adding 'apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py' adding 'apex/contrib/multihead_attn/mask_softmax_dropout_func.py' adding 'apex/contrib/multihead_attn/self_multihead_attn.py' adding 'apex/contrib/multihead_attn/self_multihead_attn_func.py' adding 'apex/contrib/optimizers/init.py' adding 'apex/contrib/optimizers/distributed_fused_adam.py' adding 'apex/contrib/optimizers/distributed_fused_lamb.py' adding 'apex/contrib/optimizers/fp16_optimizer.py' adding 'apex/contrib/optimizers/fused_adam.py' adding 'apex/contrib/optimizers/fused_lamb.py' adding 'apex/contrib/optimizers/fused_sgd.py' adding 'apex/contrib/peer_memory/init.py' adding 'apex/contrib/peer_memory/peer_halo_exchanger_1d.py' adding 'apex/contrib/peer_memory/peer_memory.py' adding 'apex/contrib/sparsity/init.py' adding 'apex/contrib/sparsity/asp.py' adding 'apex/contrib/sparsity/permutation_lib.py' adding 'apex/contrib/sparsity/sparse_masklib.py' adding 'apex/contrib/sparsity/permutation_search_kernels/init.py' adding 'apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py' adding 'apex/contrib/sparsity/permutation_search_kernels/channel_swap.py' adding 'apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py' adding 'apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py' adding 'apex/contrib/test/init.py' adding 'apex/contrib/test/bottleneck/init.py' adding 'apex/contrib/test/bottleneck/test_bottleneck_module.py' adding 'apex/contrib/test/clip_grad/init.py' adding 'apex/contrib/test/clip_grad/test_clip_grad.py' adding 'apex/contrib/test/conv_bias_relu/init.py' adding 'apex/contrib/test/conv_bias_relu/test_conv_bias_relu.py' adding 'apex/contrib/test/cudnn_gbn/init.py' adding 'apex/contrib/test/cudnn_gbn/test_cudnn_gbn_with_two_gpus.py' adding 'apex/contrib/test/fmha/init.py' adding 'apex/contrib/test/fmha/test_fmha.py' adding 'apex/contrib/test/focal_loss/init.py' adding 'apex/contrib/test/focal_loss/test_focal_loss.py' adding 'apex/contrib/test/index_mul_2d/init.py' adding 'apex/contrib/test/index_mul_2d/test_index_mul_2d.py' adding 'apex/contrib/test/layer_norm/init.py' adding 'apex/contrib/test/layer_norm/test_fast_layer_norm.py' adding 'apex/contrib/test/multihead_attn/init.py' adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn.py' adding 'apex/contrib/test/multihead_attn/test_encdec_multihead_attn_norm_add.py' adding 'apex/contrib/test/multihead_attn/test_fast_self_multihead_attn_bias.py' adding 'apex/contrib/test/multihead_attn/test_mha_fused_softmax.py' adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn.py' adding 'apex/contrib/test/multihead_attn/test_self_multihead_attn_norm_add.py' adding 'apex/contrib/test/optimizers/init.py' adding 'apex/contrib/test/optimizers/test_dist_adam.py' adding 'apex/contrib/test/optimizers/test_distributed_fused_lamb.py' adding 'apex/contrib/test/peer_memory/init.py' adding 'apex/contrib/test/peer_memory/test_peer_halo_exchange_module.py' adding 'apex/contrib/test/transducer/init.py' adding 'apex/contrib/test/transducer/test_transducer_joint.py' adding 'apex/contrib/test/transducer/test_transducer_loss.py' adding 'apex/contrib/test/xentropy/init.py' adding 'apex/contrib/test/xentropy/test_label_smoothing.py' adding 'apex/contrib/transducer/init.py' adding 'apex/contrib/transducer/_transducer_ref.py' adding 'apex/contrib/transducer/transducer.py' adding 'apex/contrib/xentropy/init.py' adding 'apex/contrib/xentropy/softmax_xentropy.py' adding 'apex/fp16_utils/init.py' adding 'apex/fp16_utils/fp16_optimizer.py' adding 'apex/fp16_utils/fp16util.py' adding 'apex/fp16_utils/loss_scaler.py' adding 'apex/fused_dense/init.py' adding 'apex/fused_dense/fused_dense.py' adding 'apex/mlp/init.py' adding 'apex/mlp/mlp.py' adding 'apex/multi_tensor_apply/init.py' adding 'apex/multi_tensor_apply/multi_tensor_apply.py' adding 'apex/normalization/init.py' adding 'apex/normalization/fused_layer_norm.py' adding 'apex/optimizers/init.py' adding 'apex/optimizers/fused_adagrad.py' adding 'apex/optimizers/fused_adam.py' adding 'apex/optimizers/fused_lamb.py' adding 'apex/optimizers/fused_mixed_precision_lamb.py' adding 'apex/optimizers/fused_novograd.py' adding 'apex/optimizers/fused_sgd.py' adding 'apex/parallel/LARC.py' adding 'apex/parallel/init.py' adding 'apex/parallel/distributed.py' adding 'apex/parallel/multiproc.py' adding 'apex/parallel/optimized_sync_batchnorm.py' adding 'apex/parallel/optimized_sync_batchnorm_kernel.py' adding 'apex/parallel/sync_batchnorm.py' adding 'apex/parallel/sync_batchnorm_kernel.py' adding 'apex/transformer/init.py' adding 'apex/transformer/_ucc_util.py' adding 'apex/transformer/enums.py' adding 'apex/transformer/log_util.py' adding 'apex/transformer/microbatches.py' adding 'apex/transformer/parallel_state.py' adding 'apex/transformer/utils.py' adding 'apex/transformer/_data/init.py' adding 'apex/transformer/_data/_batchsampler.py' adding 'apex/transformer/amp/init.py' adding 'apex/transformer/amp/grad_scaler.py' adding 'apex/transformer/functional/init.py' adding 'apex/transformer/functional/fused_softmax.py' adding 'apex/transformer/layers/init.py' adding 'apex/transformer/layers/layer_norm.py' adding 'apex/transformer/pipeline_parallel/init.py' adding 'apex/transformer/pipeline_parallel/_timers.py' adding 'apex/transformer/pipeline_parallel/p2p_communication.py' adding 'apex/transformer/pipeline_parallel/utils.py' adding 'apex/transformer/pipeline_parallel/schedules/init.py' adding 'apex/transformer/pipeline_parallel/schedules/common.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py' adding 'apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py' adding 'apex/transformer/tensor_parallel/init.py' adding 'apex/transformer/tensor_parallel/cross_entropy.py' adding 'apex/transformer/tensor_parallel/data.py' adding 'apex/transformer/tensor_parallel/layers.py' adding 'apex/transformer/tensor_parallel/mappings.py' adding 'apex/transformer/tensor_parallel/memory.py' adding 'apex/transformer/tensor_parallel/random.py' adding 'apex/transformer/tensor_parallel/utils.py' adding 'apex/transformer/testing/init.py' adding 'apex/transformer/testing/arguments.py' adding 'apex/transformer/testing/commons.py' adding 'apex/transformer/testing/distributed_test_base.py' adding 'apex/transformer/testing/global_vars.py' adding 'apex/transformer/testing/standalone_bert.py' adding 'apex/transformer/testing/standalone_gpt.py' adding 'apex/transformer/testing/standalone_transformer_lm.py' adding 'apex-0.1.dist-info/LICENSE' adding 'apex-0.1.dist-info/METADATA' adding 'apex-0.1.dist-info/WHEEL' adding 'apex-0.1.dist-info/top_level.txt' adding 'apex-0.1.dist-info/RECORD' removing build/bdist.linux-x86_64/wheel Building wheel for apex (setup.py) ... done Created wheel for apex: filename=apex-0.1-py3-none-any.whl size=346778 sha256=bac9bdfa231d1f15122a9c3f292beee94df5bcf1c979b3240091d5c90b3f269c Stored in directory: /tmp/pip-ephem-wheel-cache-6m7s92_k/wheels/0a/df/3d/18f4c8df382be1e9dc0b4f5c16593b2d47107867ca80e7a6f6 Successfully built apex Installing collected packages: apex Attempting uninstall: apex Found existing installation: apex 0.1 Uninstalling apex-0.1: Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex-0.1.dist-info/ Removing file or directory /home/cxq/.conda/envs/torch10/lib/python3.7/site-packages/apex/ Successfully uninstalled apex-0.1 Successfully installed apex-0.1

Hi, did you get any error when running: python setup.py build develop ? I got the error below

[jinhseo]

@cxq1 I found some relevant articles from https://github.com/NVIDIA/apex/issues/1532 about this issue. So try: git clone --branch 22.04-dev https://github.com/NVIDIA/apex.git for older version. I will update on readme as well.

[jinhseo]

@ShirleySyh I am not sure apex is successfully installed before u run it. Do you want to try the way I commented?

[jinhseo]

If system cannot find available GPU architecture, then try export TORCH_CUDA_ARCH_LIST="compute capability". "compute capability" for your GPU could be found at here.

[cxq1]

@cxq1 I found some relevant articles from NVIDIA/apex#1532 about this issue. So try: git clone --branch 22.04-dev https://github.com/NVIDIA/apex.git for older version. I will update on readme as well.

Thank you for your reply.