apex installation failures

command "pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./"

error log: "DEPRECATION: --build-option and --global-option are deprecated. pip 24.0 will enforce this behaviour change. A possible replacement is to use --config-settings. Discussion can be found at https://github.com/pypa/pip/issues/11859 WARNING: Implying --no-binary=:all: due to the presence of --build-option / --global-option. Processing /root/junyan/apex Running command python setup.py egg_info

torch.version = 1.9.0+cu111

running egg_info creating /tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info writing /tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info/PKG-INFO writing dependency_links to /tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info/dependency_links.txt writing top-level names to /tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info/top_level.txt writing manifest file '/tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info/SOURCES.txt' reading manifest file '/tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file '/tmp/pip-pip-egg-info-eip1b_ig/apex.egg-info/SOURCES.txt' /root/junyan/apex/setup.py:111: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies! warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!") Preparing metadata (setup.py) ... done Building wheels for collected packages: apex Running command python setup.py bdist_wheel

torch.version = 1.9.0+cu111

/root/junyan/apex/setup.py:111: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies! warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")

Compiling cuda extensions with nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Tue_Sep_15_19:10:02_PDT_2020 Cuda compilation tools, release 11.1, V11.1.74 Build cuda_11.1.TC455_06.29069683_0 from /usr/local/cuda/bin

running bdist_wheel running build running build_py creating build/lib.linux-x86_64-cpython-37 creating build/lib.linux-x86_64-cpython-37/apex copying apex/init.py -> build/lib.linux-x86_64-cpython-37/apex copying apex/_autocast_utils.py -> build/lib.linux-x86_64-cpython-37/apex creating build/lib.linux-x86_64-cpython-37/apex/RNN copying apex/RNN/init.py -> build/lib.linux-x86_64-cpython-37/apex/RNN copying apex/RNN/RNNBackend.py -> build/lib.linux-x86_64-cpython-37/apex/RNN copying apex/RNN/cells.py -> build/lib.linux-x86_64-cpython-37/apex/RNN copying apex/RNN/models.py -> build/lib.linux-x86_64-cpython-37/apex/RNN creating build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/init.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/version.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/_process_optimizer.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/opt.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/rnn_compat.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/scaler.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/utils.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/wrap.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/amp.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/frontend.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/_amp_state.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/_initialize.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/compat.py -> build/lib.linux-x86_64-cpython-37/apex/amp copying apex/amp/handle.py -> build/lib.linux-x86_64-cpython-37/apex/amp creating build/lib.linux-x86_64-cpython-37/apex/contrib copying apex/contrib/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib creating build/lib.linux-x86_64-cpython-37/apex/fp16_utils copying apex/fp16_utils/init.py -> build/lib.linux-x86_64-cpython-37/apex/fp16_utils copying apex/fp16_utils/fp16_optimizer.py -> build/lib.linux-x86_64-cpython-37/apex/fp16_utils copying apex/fp16_utils/fp16util.py -> build/lib.linux-x86_64-cpython-37/apex/fp16_utils copying apex/fp16_utils/loss_scaler.py -> build/lib.linux-x86_64-cpython-37/apex/fp16_utils creating build/lib.linux-x86_64-cpython-37/apex/multi_tensor_apply copying apex/multi_tensor_apply/init.py -> build/lib.linux-x86_64-cpython-37/apex/multi_tensor_apply copying apex/multi_tensor_apply/multi_tensor_apply.py -> build/lib.linux-x86_64-cpython-37/apex/multi_tensor_apply creating build/lib.linux-x86_64-cpython-37/apex/normalization copying apex/normalization/init.py -> build/lib.linux-x86_64-cpython-37/apex/normalization copying apex/normalization/fused_layer_norm.py -> build/lib.linux-x86_64-cpython-37/apex/normalization copying apex/normalization/instance_norm.py -> build/lib.linux-x86_64-cpython-37/apex/normalization creating build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/init.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/fused_adagrad.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/fused_adam.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/fused_lamb.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/fused_mixed_precision_lamb.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/fused_novograd.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers copying apex/optimizers/fused_sgd.py -> build/lib.linux-x86_64-cpython-37/apex/optimizers creating build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/multiproc.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/optimized_sync_batchnorm.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/sync_batchnorm_kernel.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/init.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/distributed.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/sync_batchnorm.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/LARC.py -> build/lib.linux-x86_64-cpython-37/apex/parallel copying apex/parallel/optimized_sync_batchnorm_kernel.py -> build/lib.linux-x86_64-cpython-37/apex/parallel creating build/lib.linux-x86_64-cpython-37/apex/pyprof copying apex/pyprof/init.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof creating build/lib.linux-x86_64-cpython-37/apex/reparameterization copying apex/reparameterization/reparameterization.py -> build/lib.linux-x86_64-cpython-37/apex/reparameterization copying apex/reparameterization/init.py -> build/lib.linux-x86_64-cpython-37/apex/reparameterization copying apex/reparameterization/weight_norm.py -> build/lib.linux-x86_64-cpython-37/apex/reparameterization creating build/lib.linux-x86_64-cpython-37/apex/fused_dense copying apex/fused_dense/init.py -> build/lib.linux-x86_64-cpython-37/apex/fused_dense copying apex/fused_dense/fused_dense.py -> build/lib.linux-x86_64-cpython-37/apex/fused_dense creating build/lib.linux-x86_64-cpython-37/apex/mlp copying apex/mlp/init.py -> build/lib.linux-x86_64-cpython-37/apex/mlp copying apex/mlp/mlp.py -> build/lib.linux-x86_64-cpython-37/apex/mlp creating build/lib.linux-x86_64-cpython-37/apex/transformer copying apex/transformer/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer copying apex/transformer/enums.py -> build/lib.linux-x86_64-cpython-37/apex/transformer copying apex/transformer/log_util.py -> build/lib.linux-x86_64-cpython-37/apex/transformer copying apex/transformer/microbatches.py -> build/lib.linux-x86_64-cpython-37/apex/transformer copying apex/transformer/parallel_state.py -> build/lib.linux-x86_64-cpython-37/apex/transformer copying apex/transformer/utils.py -> build/lib.linux-x86_64-cpython-37/apex/transformer creating build/lib.linux-x86_64-cpython-37/apex/amp/lists copying apex/amp/lists/init.py -> build/lib.linux-x86_64-cpython-37/apex/amp/lists copying apex/amp/lists/functional_overrides.py -> build/lib.linux-x86_64-cpython-37/apex/amp/lists copying apex/amp/lists/torch_overrides.py -> build/lib.linux-x86_64-cpython-37/apex/amp/lists copying apex/amp/lists/tensor_overrides.py -> build/lib.linux-x86_64-cpython-37/apex/amp/lists creating build/lib.linux-x86_64-cpython-37/apex/contrib/groupbn copying apex/contrib/groupbn/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/groupbn copying apex/contrib/groupbn/batch_norm.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/groupbn creating build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/encdec_multihead_attn.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/encdec_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/fast_encdec_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/fast_encdec_multihead_attn_norm_add_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/fast_self_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/fast_self_multihead_attn_norm_add_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/mask_softmax_dropout_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/self_multihead_attn.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn copying apex/contrib/multihead_attn/self_multihead_attn_func.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/multihead_attn creating build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/fp16_optimizer.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/fused_sgd.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/distributed_fused_adam.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/distributed_fused_adam_v2.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/distributed_fused_adam_v3.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/distributed_fused_lamb.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/fused_adam.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers copying apex/contrib/optimizers/fused_lamb.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/optimizers creating build/lib.linux-x86_64-cpython-37/apex/contrib/xentropy copying apex/contrib/xentropy/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/xentropy copying apex/contrib/xentropy/softmax_xentropy.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/xentropy creating build/lib.linux-x86_64-cpython-37/apex/contrib/bottleneck copying apex/contrib/bottleneck/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/bottleneck copying apex/contrib/bottleneck/bottleneck.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/bottleneck copying apex/contrib/bottleneck/bottleneck_module_test.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/bottleneck copying apex/contrib/bottleneck/halo_exchangers.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/bottleneck copying apex/contrib/bottleneck/test.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/bottleneck creating build/lib.linux-x86_64-cpython-37/apex/contrib/conv_bias_relu copying apex/contrib/conv_bias_relu/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/conv_bias_relu copying apex/contrib/conv_bias_relu/conv_bias_relu.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/conv_bias_relu creating build/lib.linux-x86_64-cpython-37/apex/contrib/fmha copying apex/contrib/fmha/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/fmha copying apex/contrib/fmha/fmha.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/fmha creating build/lib.linux-x86_64-cpython-37/apex/contrib/focal_loss copying apex/contrib/focal_loss/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/focal_loss copying apex/contrib/focal_loss/focal_loss.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/focal_loss creating build/lib.linux-x86_64-cpython-37/apex/contrib/layer_norm copying apex/contrib/layer_norm/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/layer_norm copying apex/contrib/layer_norm/layer_norm.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/layer_norm creating build/lib.linux-x86_64-cpython-37/apex/contrib/peer_memory copying apex/contrib/peer_memory/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/peer_memory copying apex/contrib/peer_memory/peer_halo_exchange_module_tests.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/peer_memory copying apex/contrib/peer_memory/peer_halo_exchanger_1d.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/peer_memory copying apex/contrib/peer_memory/peer_memory.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/peer_memory creating build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity copying apex/contrib/sparsity/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity copying apex/contrib/sparsity/asp.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity copying apex/contrib/sparsity/permutation_lib.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity copying apex/contrib/sparsity/sparse_masklib.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity creating build/lib.linux-x86_64-cpython-37/apex/contrib/transducer copying apex/contrib/transducer/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/transducer copying apex/contrib/transducer/transducer.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/transducer creating build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity/permutation_search_kernels copying apex/contrib/sparsity/permutation_search_kernels/init.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity/permutation_search_kernels copying apex/contrib/sparsity/permutation_search_kernels/call_permutation_search_kernels.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity/permutation_search_kernels copying apex/contrib/sparsity/permutation_search_kernels/exhaustive_search.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity/permutation_search_kernels copying apex/contrib/sparsity/permutation_search_kernels/permutation_utilities.py -> build/lib.linux-x86_64-cpython-37/apex/contrib/sparsity/permutation_search_kernels creating build/lib.linux-x86_64-cpython-37/apex/pyprof/nvtx copying apex/pyprof/nvtx/init.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/nvtx copying apex/pyprof/nvtx/nvmarker.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/nvtx creating build/lib.linux-x86_64-cpython-37/apex/pyprof/parse copying apex/pyprof/parse/init.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/parse copying apex/pyprof/parse/main.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/parse copying apex/pyprof/parse/db.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/parse copying apex/pyprof/parse/kernel.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/parse copying apex/pyprof/parse/nvvp.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/parse copying apex/pyprof/parse/parse.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/parse creating build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/main.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/activation.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/base.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/blas.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/conv.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/convert.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/data.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/dropout.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/embedding.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/index_slice_join_mutate.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/linear.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/loss.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/misc.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/normalization.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/optim.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/output.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/pointwise.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/pooling.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/prof.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/randomSample.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/recurrentCell.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/reduction.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/softmax.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/usage.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/init.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof copying apex/pyprof/prof/utility.py -> build/lib.linux-x86_64-cpython-37/apex/pyprof/prof creating build/lib.linux-x86_64-cpython-37/apex/transformer/_data copying apex/transformer/_data/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/_data copying apex/transformer/_data/_batchsampler.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/_data creating build/lib.linux-x86_64-cpython-37/apex/transformer/amp copying apex/transformer/amp/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/amp copying apex/transformer/amp/grad_scaler.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/amp creating build/lib.linux-x86_64-cpython-37/apex/transformer/functional copying apex/transformer/functional/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/functional copying apex/transformer/functional/fused_softmax.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/functional creating build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel copying apex/transformer/pipeline_parallel/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel copying apex/transformer/pipeline_parallel/_timers.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel copying apex/transformer/pipeline_parallel/p2p_communication.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel copying apex/transformer/pipeline_parallel/utils.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel creating build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/cross_entropy.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/data.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/layers.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/mappings.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/memory.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/random.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel copying apex/transformer/tensor_parallel/utils.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/tensor_parallel creating build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/arguments.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/commons.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/distributed_test_base.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/global_vars.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/standalone_bert.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing copying apex/transformer/testing/standalone_gpt.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/testing creating build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel/schedules copying apex/transformer/pipeline_parallel/schedules/init.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel/schedules copying apex/transformer/pipeline_parallel/schedules/common.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel/schedules copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_no_pipelining.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel/schedules copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_with_interleaving.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel/schedules copying apex/transformer/pipeline_parallel/schedules/fwd_bwd_pipelining_without_interleaving.py -> build/lib.linux-x86_64-cpython-37/apex/transformer/pipeline_parallel/schedules running build_ext building 'apex_C' extension creating /root/junyan/apex/build/temp.linux-x86_64-cpython-37 creating /root/junyan/apex/build/temp.linux-x86_64-cpython-37/csrc Traceback (most recent call last): File "", line 36, in File "", line 34, in File "/root/junyan/apex/setup.py", line 712, in extras_require=extras, File "/opt/conda/lib/python3.7/site-packages/setuptools/init.py", line 107, in setup return distutils.core.setup(attrs) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/opt/conda/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command super().run_command(command) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/opt/conda/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 325, in run self.run_command("build") File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/opt/conda/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command super().run_command(command) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/command/build.py", line 131, in run self.run_command(cmd_name) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/opt/conda/lib/python3.7/site-packages/setuptools/dist.py", line 1234, in run_command super().run_command(command) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 84, in run _build_ext.run(self) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run self.build_extensions() File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 709, in build_extensions build_ext.build_extensions(self) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions self._build_extensions_serial() File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial self.build_extension(ext) File "/opt/conda/lib/python3.7/site-packages/setuptools/command/build_ext.py", line 246, in build_extension _build_ext.build_extension(self, ext) File "/opt/conda/lib/python3.7/site-packages/setuptools/_distutils/command/build_ext.py", line 555, in build_extension depends=ext.depends, File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 539, in unix_wrap_ninja_compile with_cuda=with_cuda) File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1336, in _write_ninja_file_and_compile_objects check_compiler_abi_compatibility(compiler) File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 283, in check_compiler_abi_compatibility if not check_compiler_ok_for_platform(compiler): File "/opt/conda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 243, in check_compiler_ok_for_platform which = subprocess.check_output(['which', compiler], stderr=subprocess.STDOUT) File "/opt/conda/lib/python3.7/subprocess.py", line 411, in check_output kwargs).stdout File "/opt/conda/lib/python3.7/subprocess.py", line 512, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['which', 'c++']' returned non-zero exit status 1. error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

except ImportError as error: print( "ERROR: Can not execute setup.py since setuptools is not available in " "the build environment.", file=sys.stderr, ) sys.exit(1)

file = %r sys.argv[0] = file

if os.path.exists(file): filename = file with tokenize.open(file) as f: setup_py_code = f.read() else: filename = "" setup_py_code = "from setuptools import setup; setup()"

exec(compile(setup_py_code, filename, "exec")) '"'"''"'"''"'"' % ('"'"'/root/junyan/apex/setup.py'"'"',), "", "exec"))' --cpp_ext --cuda_ext bdist_wheel -d /tmp/pip-wheel-53qg6he5 cwd: /root/junyan/apex/ Building wheel for apex (setup.py) ... error ERROR: Failed building wheel for apex Running setup.py clean for apex Running command python setup.py clean

torch.version = 1.9.0+cu111

/root/junyan/apex/setup.py:111: UserWarning: Option --pyprof not specified. Not installing PyProf dependencies! warnings.warn("Option --pyprof not specified. Not installing PyProf dependencies!")

running clean removing 'build/temp.linux-x86_64-cpython-37' (and everything under it) removing 'build/lib.linux-x86_64-cpython-37' (and everything under it) 'build/bdist.linux-x86_64' does not exist -- can't clean it 'build/scripts-3.7' does not exist -- can't clean it Failed to build apex ERROR: Could not build wheels for apex, which is required to install pyproject.toml-based projects ". It is very weird, but the installation process is blocked.

Could you help me?

Best Regards! Momo

NVIDIA / apex

apex installation failures #1764