An error is reported when running python setup_cuda.py install

[linuxdevopscn]

(textgen) quanlian@quanlian-System-Product-Name:~/aigc/text-generation-webui/repositories/GPTQ-for-LLaMa$ python setup_cuda.py install running install /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. !!

    ********************************************************************************
    Please avoid running ``setup.py`` directly.
    Instead, use pypa/build, pypa/installer, pypa/build or
    other standards-based tools.

    See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
    ********************************************************************************

!! self.initialize_options() /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated. !!

    ********************************************************************************
    Please avoid running ``setup.py`` and ``easy_install``.
    Instead, use pypa/build, pypa/installer, pypa/build or
    other standards-based tools.

    See https://github.com/pypa/setuptools/issues/917 for details.
    ********************************************************************************

!! self.initialize_options() running bdist_egg running egg_info creating quant_cuda.egg-info writing quant_cuda.egg-info/PKG-INFO writing dependency_links to quant_cuda.egg-info/dependency_links.txt writing top-level names to quant_cuda.egg-info/top_level.txt writing manifest file 'quant_cuda.egg-info/SOURCES.txt' /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/utils/cpp_extension.py:477: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.')) reading manifest file 'quant_cuda.egg-info/SOURCES.txt' writing manifest file 'quant_cuda.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_ext /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/utils/cpp_extension.py:399: UserWarning: There are no g++ version bounds defined for CUDA version 12.1 warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}') building 'quant_cuda' extension creating build creating build/temp.linux-x86_64-cpython-310 gcc -pthread -B /home/quanlian/mambaforge/envs/textgen/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/quanlian/mambaforge/envs/textgen/include -fPIC -O2 -isystem /home/quanlian/mambaforge/envs/textgen/include -fPIC -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/TH -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda-12.1/include -I/home/quanlian/mambaforge/envs/textgen/include/python3.10 -c quant_cuda.cpp -o build/temp.linux-x86_64-cpython-310/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 /usr/local/cuda-12.1/bin/nvcc -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/TH -I/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda-12.1/include -I/home/quanlian/mambaforge/envs/textgen/include/python3.10 -c quant_cuda_kernel.cu -o build/temp.linux-x86_64-cpython-310/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++17 /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/pybind11/detail/../cast.h: In function ‘typename pybind11::detail::type_caster<typename pybind11::detail::intrinsic_type::type>::cast_op_type pybind11::detail::cast_op(make_caster&)’: /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/pybind11/detail/../cast.h:42:120: error: expected template-name before ‘<’ token 42 | return caster.operator typename make_caster::template cast_op_type(); | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/pybind11/detail/../cast.h:42:120: error: expected identifier before ‘<’ token /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/pybind11/detail/../cast.h:42:123: error: expected primary-expression before ‘>’ token 42 | return caster.operator typename make_caster::template cast_op_type(); | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/pybind11/detail/../cast.h:42:126: error: expected primary-expression before ‘)’ token 42 | return caster.operator typename make_caster::template cast_op_type(); | ^ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:128:40: warning: ‘at::DeprecatedTypeProperties& at::Tensor::type() const’ is deprecated: Tensor.type() is deprecated. Instead use Tensor.options(), which in many cases (e.g. in a constructor) is a drop-in replacement. If you were using data from type(), that is now available from Tensor itself, so instead of tensor.type().scalar_type(), use tensor.scalar_type() instead and instead of tensor.type().backend() use tensor.device(). [-Wdeprecated-declarations] 128 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:223:1: note: declared here 223 | DeprecatedTypeProperties & type() const { | ^ ~~ quant_cuda_kernel.cu:128:154: warning: ‘c10::ScalarType detail::scalar_type(const at::DeprecatedTypeProperties&)’ is deprecated: passing at::DeprecatedTypeProperties to an AT_DISPATCH macro is deprecated, pass an at::ScalarType instead [-Wdeprecated-declarations] 128 | AT_DISPATCH_FLOATING_TYPES( | ^
/home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/Dispatch.h:109:1: note: declared here 109 | inline at::ScalarType scalar_type(const at::DeprecatedTypeProperties& t) { | ^~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:128:1010: warning: ‘T at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 128 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:128:1031: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 128 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ .... /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:215:1105: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 215 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:215:1128: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 215 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T * data() const { | ^ ~~

quant_cuda_kernel.cu:448:1082: warning: ‘T at::Tensor::data() const [with T = double]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:1105: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:1128: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu: In lambda function: quant_cuda_kernel.cu:448:2010: warning: ‘T at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:2031: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:2054: warning: ‘T at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:2080: warning: ‘T at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:2103: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ quant_cuda_kernel.cu:448:2126: warning: ‘T at::Tensor::data() const [with T = int]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations] 448 | AT_DISPATCH_FLOATING_TYPES( | ^ /home/quanlian/mambaforge/envs/textgen/lib/python3.10/site-packages/torch/include/ATen/core/TensorBody.h:245:1: note: declared here 245 | T data() const { | ^ ~~ error: command '/usr/local/cuda-12.1/bin/nvcc' failed with exit code 1

[linuxdevopscn]

[globavi]

Hi @linuxdevopscn, I get same error with 525. Did you come up with solution yet for that?