search

hpcaitech / EnergonAI

Large-scale model inference.
Apache License 2.0
631 stars 90 forks source link

fail to install EnergonAI #204

Open NewDriverLee opened 1 year ago

NewDriverLee commented 1 year ago

I use anaconda, python 3.10 and pytorch 1.13.1 .

When I ran the following Installation command: pip install . an error happened. Part of the error message is:

Processing /home/liwj/project/EnergonAI_github
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: energonai
  Building wheel for energonai (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [113 lines of output]

      torch.__version__  = 1.13.1

      Compiling cuda extensions with
      nvcc: NVIDIA (R) Cuda compiler driver
      Copyright (c) 2005-2022 NVIDIA Corporation
      Built on Tue_Mar__8_18:18:20_PST_2022
      Cuda compilation tools, release 11.6, V11.6.124
      Build cuda_11.6.r11.6/compiler.31057947_0
      from /home/liwj/miniconda3/envs/py3.10/bin

      running bdist_wheel
      running build
      running build_py
      running build_ext
      building 'energonai_scale_mask' extension
      Emitting ninja build file /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      ninja: no work to do.
      g++ -pthread -B /home/liwj/miniconda3/envs/py3.10/compiler_compat -shared -Wl,-rpath,/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath-link,/home/liwj/miniconda3/envs/py3.10/lib -L/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath,/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath-link,/home/liwj/miniconda3/envs/py3.10/lib -L/home/liwj/miniconda3/envs/py3.10/lib /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o -L/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/lib -L/home/liwj/miniconda3/envs/py3.10/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda_cu -ltorch_cuda_cpp -o build/lib.linux-x86_64-cpython-310/energonai_scale_mask.cpython-310-x86_64-linux-gnu.so
      building 'energonai_layer_norm' extension
      Emitting ninja build file /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      [1/1] /home/liwj/miniconda3/envs/py3.10/bin/nvcc  -I/home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/TH -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/THC -I/home/liwj/miniconda3/envs/py3.10/include -I/home/liwj/miniconda3/envs/py3.10/include/python3.10 -c -c /home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -o /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTHRUST_IGNORE_CUB_VERSION_CHECK -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_layer_norm -D_GLIBCXX_USE_CXX11_ABI=0
      FAILED: /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.o
      /home/liwj/miniconda3/envs/py3.10/bin/nvcc  -I/home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/TH -I/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/THC -I/home/liwj/miniconda3/envs/py3.10/include -I/home/liwj/miniconda3/envs/py3.10/include/python3.10 -c -c /home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu -o /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 --use_fast_math -DVERSION_GE_1_1 -DVERSION_GE_1_3 -DVERSION_GE_1_5 -DUSE_C10D_NCCL -std=c++14 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -U__CUDA_NO_HALF2_OPERATORS__ -DTHRUST_IGNORE_CUB_VERSION_CHECK -gencode arch=compute_70,code=sm_70 -gencode arch=compute_80,code=sm_80 --threads 4 -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=energonai_layer_norm -D_GLIBCXX_USE_CXX11_ABI=0
      sh: /home/liwj/miniconda3/envs/py3.10/bin/../lib/libtinfo.so.6: no version information available (required by sh)
      sh: /home/liwj/miniconda3/envs/py3.10/bin/../lib/libtinfo.so.6: no version information available (required by sh)
      sh: /home/liwj/miniconda3/envs/py3.10/bin/../lib/libtinfo.so.6: no version information available (required by sh)
      In file included from /home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu:10:
      /home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:10:10: fatal error: cusolverDn.h: No such file or directory
         10 | #include <cusolverDn.h>
            |          ^~~~~~~~~~~~~~
      compilation terminated.
      In file included from /home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu:10:
      /home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:10:10: fatal error: cusolverDn.h: No such file or directory
         10 | #include <cusolverDn.h>
            |          ^~~~~~~~~~~~~~
      compilation terminated.
      In file included from /home/liwj/project/EnergonAI_github/energonai/kernel/cuda_native/csrc/layer_norm_cuda_kernel.cu:10:
      /home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/include/ATen/cuda/CUDAContext.h:10:10: fatal error: cusolverDn.h: No such file or directory
         10 | #include <cusolverDn.h>
            |          ^~~~~~~~~~~~~~
      compilation terminated.
      ninja: build stopped: subcommand failed.
      Traceback (most recent call last):
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1900, in _run_ninja_build
          subprocess.run(
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/subprocess.py", line 526, in run
          raise CalledProcessError(retcode, process.args,
      subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

      The above exception was the direct cause of the following exception:

      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/home/liwj/project/EnergonAI_github/setup.py", line 164, in <module>
          setup(
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/__init__.py", line 108, in setup
          return distutils.core.setup(**attrs)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
          return run_commands(dist)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
          dist.run_commands()
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 325, in run
          self.run_command("build")
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/dist.py", line 1221, in run_command
          super().run_command(command)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 84, in run
          _build_ext.run(self)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
          build_ext.build_extensions(self)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
          self._build_extensions_serial()
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
          self.build_extension(ext)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
          _build_ext.build_extension(self, ext)
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
          objects = self.compiler.compile(
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
          _write_ninja_file_and_compile_objects(
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1573, in _write_ninja_file_and_compile_objects
          _run_ninja_build(
        File "/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1916, in _run_ninja_build
          raise RuntimeError(message) from e
      RuntimeError: Error compiling objects for extension
      [end of output]
LemonSqi commented 1 year ago

Try to modify the content of line 1865 in cpp_extension.py is: command=['ninja ',' -- version '] This may be an exception caused by python compatibility issues

NewDriverLee commented 1 year ago

Try to modify the content of line 1865 in cpp_extension.py is: command=['ninja ',' -- version '] This may be an exception caused by python compatibility issues

Thank you very much for your reply. I have modified the cpp_extension.py as you suggested but a new error arose. The error information is as follows. It seems that it did not find the two files "scale_mask_softmax_kernel.o" and "scale_mask_softmax_wrapper.o" in the directory "/home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc". Actually, I have checked that there was no file in such directory.

Processing /home/liwj/project/EnergonAI_github
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: energonai
  Building wheel for energonai (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [83 lines of output]

      torch.__version__  = 1.13.1

      Compiling cuda extensions with
      nvcc: NVIDIA (R) Cuda compiler driver
      Copyright (c) 2005-2022 NVIDIA Corporation
      Built on Tue_Mar__8_18:18:20_PST_2022
      Cuda compilation tools, release 11.6, V11.6.124
      Build cuda_11.6.r11.6/compiler.31057947_0
      from /home/liwj/miniconda3/envs/py3.10/bin

      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/batch_mgr.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/engine.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/pipe.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/task.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/worker.py -> build/lib.linux-x86_64-cpython-310/energonai
      creating build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/collective.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/p2p.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/ring.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/utils.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      creating build/lib.linux-x86_64-cpython-310/energonai/kernel
      copying energonai/kernel/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel
      creating build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/dynamic_batch_manager.py -> build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/naive_batch_manager.py -> build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      creating build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/attention.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/downstream.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/embedding.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/endecoder.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/mlp.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/model_factory.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      creating build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/energon_tracer.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/split_method.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/split_policy.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      creating build/lib.linux-x86_64-cpython-310/energonai/testing
      copying energonai/testing/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/testing
      copying energonai/testing/models.py -> build/lib.linux-x86_64-cpython-310/energonai/testing
      creating build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/checkpointing.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/checkpointing_hf_gpt2.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/checkpointing_opt.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/common.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/files.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/timer.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      creating build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/layer_norm.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/linear_func.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/scale_mask_softmax.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/transpose_pad.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      running build_ext
      building 'energonai_scale_mask' extension
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc
      Emitting ninja build file /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      1.11.1.git.kitware.jobserver-1
      g++ -pthread -B /home/liwj/miniconda3/envs/py3.10/compiler_compat -shared -Wl,-rpath,/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath-link,/home/liwj/miniconda3/envs/py3.10/lib -L/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath,/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath-link,/home/liwj/miniconda3/envs/py3.10/lib -L/home/liwj/miniconda3/envs/py3.10/lib /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o -L/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/lib -L/home/liwj/miniconda3/envs/py3.10/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda_cu -ltorch_cuda_cpp -o build/lib.linux-x86_64-cpython-310/energonai_scale_mask.cpython-310-x86_64-linux-gnu.so
      g++: error: /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o: No such file or directory
      g++: error: /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o: No such file or directory
      error: command '/usr/bin/g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for energonai
  Running setup.py clean for energonai
Failed to build energonai
Installing collected packages: energonai
  Running setup.py install for energonai ... error
  error: subprocess-exited-with-error

  × Running setup.py install for energonai did not run successfully.
  │ exit code: 1
  ╰─> [85 lines of output]

      torch.__version__  = 1.13.1

      Compiling cuda extensions with
      nvcc: NVIDIA (R) Cuda compiler driver
      Copyright (c) 2005-2022 NVIDIA Corporation
      Built on Tue_Mar__8_18:18:20_PST_2022
      Cuda compilation tools, release 11.6, V11.6.124
      Build cuda_11.6.r11.6/compiler.31057947_0
      from /home/liwj/miniconda3/envs/py3.10/bin

      running install
      /home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/batch_mgr.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/engine.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/pipe.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/task.py -> build/lib.linux-x86_64-cpython-310/energonai
      copying energonai/worker.py -> build/lib.linux-x86_64-cpython-310/energonai
      creating build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/collective.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/p2p.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/ring.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      copying energonai/communication/utils.py -> build/lib.linux-x86_64-cpython-310/energonai/communication
      creating build/lib.linux-x86_64-cpython-310/energonai/kernel
      copying energonai/kernel/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel
      creating build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/dynamic_batch_manager.py -> build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      copying energonai/legacy_batch_mgr/naive_batch_manager.py -> build/lib.linux-x86_64-cpython-310/energonai/legacy_batch_mgr
      creating build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/attention.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/downstream.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/embedding.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/endecoder.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/mlp.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      copying energonai/model/model_factory.py -> build/lib.linux-x86_64-cpython-310/energonai/model
      creating build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/energon_tracer.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/split_method.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      copying energonai/pipelinable/split_policy.py -> build/lib.linux-x86_64-cpython-310/energonai/pipelinable
      creating build/lib.linux-x86_64-cpython-310/energonai/testing
      copying energonai/testing/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/testing
      copying energonai/testing/models.py -> build/lib.linux-x86_64-cpython-310/energonai/testing
      creating build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/checkpointing.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/checkpointing_hf_gpt2.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/checkpointing_opt.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/common.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/files.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      copying energonai/utils/timer.py -> build/lib.linux-x86_64-cpython-310/energonai/utils
      creating build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/__init__.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/layer_norm.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/linear_func.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/scale_mask_softmax.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      copying energonai/kernel/cuda_native/transpose_pad.py -> build/lib.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      running build_ext
      building 'energonai_scale_mask' extension
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native
      creating /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc
      Emitting ninja build file /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/build.ninja...
      Compiling objects...
      Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
      1.11.1.git.kitware.jobserver-1
      g++ -pthread -B /home/liwj/miniconda3/envs/py3.10/compiler_compat -shared -Wl,-rpath,/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath-link,/home/liwj/miniconda3/envs/py3.10/lib -L/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath,/home/liwj/miniconda3/envs/py3.10/lib -Wl,-rpath-link,/home/liwj/miniconda3/envs/py3.10/lib -L/home/liwj/miniconda3/envs/py3.10/lib /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o -L/home/liwj/miniconda3/envs/py3.10/lib/python3.10/site-packages/torch/lib -L/home/liwj/miniconda3/envs/py3.10/lib -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda_cu -ltorch_cuda_cpp -o build/lib.linux-x86_64-cpython-310/energonai_scale_mask.cpython-310-x86_64-linux-gnu.so
      g++: error: /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_kernel.o: No such file or directory
      g++: error: /home/liwj/project/EnergonAI_github/build/temp.linux-x86_64-cpython-310/energonai/kernel/cuda_native/csrc/scale_mask_softmax_wrapper.o: No such file or directory
      error: command '/usr/bin/g++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> energonai

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
ver217 commented 1 year ago

Could you provide more information about your OS, CUDA version and gcc version? By the way, you can run pip install -v . to get detail output.