search

johnsmith0031 / alpaca_lora_4bit

MIT License
533 stars 86 forks source link

trying to get this working with text-generation-webui #154

Open caterpillarpants opened 8 months ago

caterpillarpants commented 8 months ago

I cannot compile the wheels. whether I use winglian or old_compatible, I end up with this error message. ` RuntimeError: Error compiling objects for extension [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for alpaca-lora-4bit Running setup.py clean for alpaca-lora-4bit Failed to build alpaca-lora-4bit Installing collected packages: alpaca-lora-4bit Running setup.py install for alpaca-lora-4bit ... error error: subprocess-exited-with-error

× Running setup.py install for alpaca-lora-4bit did not run successfully. │ exit code: 1 ╰─> [125 lines of output] `

then it resumes installing and later gives this error. ` RuntimeError: Error compiling objects for extension [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> alpaca-lora-4bit `

I had gotten this to work in the past, but have since had to reinstall text-generation-webui, at which time my "monkey-patch" stopped working. Any ideas to get it to work? I need the 4bit support since I have limited vram.

johnsmith0031 commented 8 months ago

Can you provide some more logs about the error?

caterpillarpants commented 8 months ago

This is my attempt to install winglian

`ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip uninstall alpaca_lora_4bit WARNING: Skipping alpaca_lora_4bit as it is not installed. ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ git checkout winglian-setup_pip Already on 'winglian-setup_pip' Your branch is up to date with 'origin/winglian-setup_pip'. ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip install . Defaulting to user installation because normal site-packages is not writeable Processing /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit Preparing metadata (setup.py) ... done Requirement already satisfied: accelerate>=0.22.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.24.0) Requirement already satisfied: colorama in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.4.6) Requirement already satisfied: datasets in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.10.1) Requirement already satisfied: einops in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.6.1) Requirement already satisfied: packaging in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (23.2) Requirement already satisfied: peft>=0.5.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.5.0) Requirement already satisfied: pyzmq in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (25.1.0) Requirement already satisfied: safetensors in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.3.1) Requirement already satisfied: sentencepiece in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.1.99) Requirement already satisfied: torch in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.0.1+cu118) Requirement already satisfied: transformers>=4.31.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (4.35.0.dev0) Requirement already satisfied: wandb in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.15.4) Requirement already satisfied: huggingface-hub in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (0.17.3) Requirement already satisfied: psutil in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.9.0) Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.4.1) Requirement already satisfied: numpy>=1.17 in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (1.24.0) Requirement already satisfied: tqdm in /home/ana/.local/lib/python3.10/site-packages (from peft>=0.5.0->alpaca-lora-4bit==0.1.3) (4.65.0) Requirement already satisfied: filelock in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.12.4) Requirement already satisfied: jinja2 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1.2) Requirement already satisfied: triton==2.0.0 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (2.0.0) Requirement already satisfied: sympy in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (1.11.1) Requirement already satisfied: typing-extensions in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (4.5.0) Requirement already satisfied: networkx in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1) Requirement already satisfied: lit in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (16.0.2) Requirement already satisfied: cmake in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (3.26.3) Requirement already satisfied: regex!=2019.12.17 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2023.3.23) Requirement already satisfied: tokenizers<0.15,>=0.14 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (0.14.1) Requirement already satisfied: requests in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2.31.0) Requirement already satisfied: multiprocess in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.70.14) Requirement already satisfied: pyarrow>=6.0.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (11.0.0) Requirement already satisfied: fsspec[http]>=2021.11.1 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2023.4.0) Requirement already satisfied: aiohttp in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.8.4) Requirement already satisfied: pandas in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2.0.3) Requirement already satisfied: responses<0.19 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.18.0) Requirement already satisfied: xxhash in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.2.0) Requirement already satisfied: dill<0.3.7,>=0.3.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.3.6) Requirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.1.31) Requirement already satisfied: appdirs>=1.4.3 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.4.4) Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (8.0.3) Requirement already satisfied: sentry-sdk>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.26.0) Requirement already satisfied: docker-pycreds>=0.4.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.4.0) Requirement already satisfied: pathtools in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.1.2) Requirement already satisfied: setproctitle in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.2.2) Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (59.6.0) Requirement already satisfied: protobuf!=4.21.0,<5,>=3.19.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.20.2) Requirement already satisfied: six>=1.4.0 in /usr/lib/python3/dist-packages (from docker-pycreds>=0.4.0->wandb->alpaca-lora-4bit==0.1.3) (1.16.0) Requirement already satisfied: attrs>=17.3.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (23.1.0) Requirement already satisfied: aiosignal>=1.1.2 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.1) Requirement already satisfied: multidict<7.0,>=4.5 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (6.0.4) Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (3.1.0) Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (4.0.2) Requirement already satisfied: frozenlist>=1.1.1 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.3) Requirement already satisfied: yarl<2.0,>=1.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.9.1) Requirement already satisfied: gitdb<5,>=4.0.1 in /home/ana/.local/lib/python3.10/site-packages (from GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (4.0.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ana/.local/lib/python3.10/site-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (1.26.16) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2020.6.20) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (3.3) Requirement already satisfied: MarkupSafe>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from jinja2->torch->alpaca-lora-4bit==0.1.3) (2.1.3) Requirement already satisfied: tzdata>=2022.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3) Requirement already satisfied: pytz>=2020.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3) Requirement already satisfied: python-dateutil>=2.8.2 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2.8.2) Requirement already satisfied: mpmath>=0.19 in /home/ana/.local/lib/python3.10/site-packages (from sympy->torch->alpaca-lora-4bit==0.1.3) (1.3.0) Requirement already satisfied: smmap<6,>=3.0.1 in /home/ana/.local/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (5.0.0) Building wheels for collected packages: alpaca-lora-4bit Building wheel for alpaca-lora-4bit (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [92 lines of output] running bdist_wheel running build running build_py package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file) running build_ext /home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem. warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda)) building 'alpaca_lora_4bit.quant_cuda' extension Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/1] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero detected during: instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" (61): here instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for alpaca-lora-4bit Running setup.py clean for alpaca-lora-4bit Failed to build alpaca-lora-4bit Installing collected packages: alpaca-lora-4bit Running setup.py install for alpaca-lora-4bit ... error error: subprocess-exited-with-error

× Running setup.py install for alpaca-lora-4bit did not run successfully. │ exit code: 1 ╰─> [125 lines of output] running install /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build/lib.linux-x86_64-3.10 creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/Finetune4bConfig.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/autograd_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/model_attn_mlp_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/matmul_utils_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/arg_parser.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/train_data.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/models.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/custom_autotune.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/triton_utils.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/gradient_checkpointing.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/amp_wrapper.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/llama_flash_attn_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/peft_tuners_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/gptq_for_llala_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/llama_attn_hijack_xformers.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server copying src/alpaca_lora_4bit/server/server.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server copying src/alpaca_lora_4bit/server/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file) running build_ext /home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem. warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda)) building 'alpaca_lora_4bit.quant_cuda' extension creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10 creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/2] c++ -MMD -MF /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda.cpp -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 [2/2] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero detected during: instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" (61): here instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/usr/lib/python3.10/distutils/command/install.py", line 619, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> alpaca-lora-4bit

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure. `

caterpillarpants commented 8 months ago

And here is my attempt to install the old_compatible.

`ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip uninstall alpaca_lora_4bit WARNING: Skipping alpaca_lora_4bit as it is not installed. ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ git checkout old_compatible Already on 'old_compatible' Your branch is up to date with 'origin/old_compatible'. ana@durga:~/text-generation-webui-main/repositories/alpaca_lora_4bit$ pip install . Defaulting to user installation because normal site-packages is not writeable Processing /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit Preparing metadata (setup.py) ... done Requirement already satisfied: accelerate>=0.22.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.24.0) Requirement already satisfied: colorama in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.4.6) Requirement already satisfied: datasets in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.10.1) Requirement already satisfied: einops in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.6.1) Requirement already satisfied: packaging in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (23.2) Requirement already satisfied: peft>=0.5.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.5.0) Requirement already satisfied: pyzmq in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (25.1.0) Requirement already satisfied: safetensors in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.3.1) Requirement already satisfied: sentencepiece in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.1.99) Requirement already satisfied: torch in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (2.0.1+cu118) Requirement already satisfied: transformers>=4.31.0 in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (4.35.0.dev0) Requirement already satisfied: wandb in /home/ana/.local/lib/python3.10/site-packages (from alpaca-lora-4bit==0.1.3) (0.15.4) Requirement already satisfied: numpy>=1.17 in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (1.24.0) Requirement already satisfied: psutil in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.9.0) Requirement already satisfied: huggingface-hub in /home/ana/.local/lib/python3.10/site-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (0.17.3) Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (from accelerate>=0.22.0->alpaca-lora-4bit==0.1.3) (5.4.1) Requirement already satisfied: tqdm in /home/ana/.local/lib/python3.10/site-packages (from peft>=0.5.0->alpaca-lora-4bit==0.1.3) (4.65.0) Requirement already satisfied: jinja2 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1.2) Requirement already satisfied: triton==2.0.0 in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (2.0.0) Requirement already satisfied: filelock in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.12.4) Requirement already satisfied: sympy in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (1.11.1) Requirement already satisfied: networkx in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (3.1) Requirement already satisfied: typing-extensions in /home/ana/.local/lib/python3.10/site-packages (from torch->alpaca-lora-4bit==0.1.3) (4.5.0) Requirement already satisfied: lit in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (16.0.2) Requirement already satisfied: cmake in /home/ana/.local/lib/python3.10/site-packages (from triton==2.0.0->torch->alpaca-lora-4bit==0.1.3) (3.26.3) Requirement already satisfied: tokenizers<0.15,>=0.14 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (0.14.1) Requirement already satisfied: regex!=2019.12.17 in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2023.3.23) Requirement already satisfied: requests in /home/ana/.local/lib/python3.10/site-packages (from transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2.31.0) Requirement already satisfied: pandas in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2.0.3) Requirement already satisfied: xxhash in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.2.0) Requirement already satisfied: responses<0.19 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.18.0) Requirement already satisfied: fsspec[http]>=2021.11.1 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (2023.4.0) Requirement already satisfied: dill<0.3.7,>=0.3.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.3.6) Requirement already satisfied: multiprocess in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (0.70.14) Requirement already satisfied: pyarrow>=6.0.0 in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (11.0.0) Requirement already satisfied: aiohttp in /home/ana/.local/lib/python3.10/site-packages (from datasets->alpaca-lora-4bit==0.1.3) (3.8.4) Requirement already satisfied: sentry-sdk>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.26.0) Requirement already satisfied: docker-pycreds>=0.4.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.4.0) Requirement already satisfied: pathtools in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (0.1.2) Requirement already satisfied: Click!=8.0.0,>=7.0 in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (8.0.3) Requirement already satisfied: protobuf!=4.21.0,<5,>=3.19.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.20.2) Requirement already satisfied: GitPython!=3.1.29,>=1.0.0 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (3.1.31) Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (59.6.0) Requirement already satisfied: appdirs>=1.4.3 in /home/ana/.local/lib/python3.10/site-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.4.4) Requirement already satisfied: setproctitle in /usr/lib/python3/dist-packages (from wandb->alpaca-lora-4bit==0.1.3) (1.2.2) Requirement already satisfied: six>=1.4.0 in /usr/lib/python3/dist-packages (from docker-pycreds>=0.4.0->wandb->alpaca-lora-4bit==0.1.3) (1.16.0) Requirement already satisfied: multidict<7.0,>=4.5 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (6.0.4) Requirement already satisfied: aiosignal>=1.1.2 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.1) Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (4.0.2) Requirement already satisfied: yarl<2.0,>=1.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.9.1) Requirement already satisfied: frozenlist>=1.1.1 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (1.3.3) Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (3.1.0) Requirement already satisfied: attrs>=17.3.0 in /home/ana/.local/lib/python3.10/site-packages (from aiohttp->datasets->alpaca-lora-4bit==0.1.3) (23.1.0) Requirement already satisfied: gitdb<5,>=4.0.1 in /home/ana/.local/lib/python3.10/site-packages (from GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (4.0.10) Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ana/.local/lib/python3.10/site-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (1.26.16) Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (3.3) Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3/dist-packages (from requests->transformers>=4.31.0->alpaca-lora-4bit==0.1.3) (2020.6.20) Requirement already satisfied: MarkupSafe>=2.0 in /home/ana/.local/lib/python3.10/site-packages (from jinja2->torch->alpaca-lora-4bit==0.1.3) (2.1.3) Requirement already satisfied: python-dateutil>=2.8.2 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2.8.2) Requirement already satisfied: tzdata>=2022.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3) Requirement already satisfied: pytz>=2020.1 in /home/ana/.local/lib/python3.10/site-packages (from pandas->datasets->alpaca-lora-4bit==0.1.3) (2023.3) Requirement already satisfied: mpmath>=0.19 in /home/ana/.local/lib/python3.10/site-packages (from sympy->torch->alpaca-lora-4bit==0.1.3) (1.3.0) Requirement already satisfied: smmap<6,>=3.0.1 in /home/ana/.local/lib/python3.10/site-packages (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->alpaca-lora-4bit==0.1.3) (5.0.0) Building wheels for collected packages: alpaca-lora-4bit Building wheel for alpaca-lora-4bit (setup.py) ... error error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [93 lines of output] running bdist_wheel running build running build_py package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file) running build_ext /home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem. warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda)) building 'alpaca_lora_4bit.quant_cuda' extension Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/2] c++ -MMD -MF /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda.cpp -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 [2/2] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero detected during: instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" (61): here instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for alpaca-lora-4bit Running setup.py clean for alpaca-lora-4bit Failed to build alpaca-lora-4bit Installing collected packages: alpaca-lora-4bit Running setup.py install for alpaca-lora-4bit ... error error: subprocess-exited-with-error

× Running setup.py install for alpaca-lora-4bit did not run successfully. │ exit code: 1 ╰─> [125 lines of output] running install /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py creating build creating build/lib.linux-x86_64-3.10 creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/Finetune4bConfig.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/autograd_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/model_attn_mlp_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/matmul_utils_4bit.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/arg_parser.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/train_data.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/models.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/custom_autotune.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/triton_utils.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/gradient_checkpointing.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit copying src/alpaca_lora_4bit/amp_wrapper.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/llama_flash_attn_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/peft_tuners_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/gptq_for_llala_lora_monkey_patch.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch copying src/alpaca_lora_4bit/monkeypatch/llama_attn_hijack_xformers.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/monkeypatch creating build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server copying src/alpaca_lora_4bit/server/server.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server copying src/alpaca_lora_4bit/server/init.py -> build/lib.linux-x86_64-3.10/alpaca_lora_4bit/server package init file 'src/alpaca_lora_4bit/quant_cuda/init.py' not found (or not a regular file) running build_ext /home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:388: UserWarning: The detected CUDA version (11.5) has a minor version mismatch with the version that was used to compile PyTorch (11.8). Most likely this shouldn't be a problem. warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda)) building 'alpaca_lora_4bit.quant_cuda' extension creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10 creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit creating /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda Emitting ninja build file /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/build.ninja... Compiling objects... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/2] c++ -MMD -MF /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o.d -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda.cpp -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda.o -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 [2/2] /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 FAILED: /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o /usr/bin/nvcc -I/home/ana/.local/lib/python3.10/site-packages/torch/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/ana/.local/lib/python3.10/site-packages/torch/include/TH -I/home/ana/.local/lib/python3.10/site-packages/torch/include/THC -I/usr/include/python3.10 -c -c /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu -o /home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/build/temp.linux-x86_64-3.10/src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.o -DCUDA_NO_HALF_OPERATORS -DCUDA_NO_HALF_CONVERSIONS -DCUDA_NO_BFLOAT16_CONVERSIONS -DCUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=quant_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_75,code=compute_75 -gencode=arch=compute_75,code=sm_75 -std=c++17 /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero detected during: instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator==(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" (61): here instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, >::operator!=(const c10::detail::integer_iterator<I, one_sided, > &) const [with I=size_t, one_sided=false, =0]" /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/core/TensorImpl.h(77): here

  /home/ana/.local/lib/python3.10/site-packages/torch/include/c10/util/irange.h(54): warning #186-D: pointless comparison of unsigned integer with zero
            detected during:
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator==(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  (61): here
              instantiation of "__nv_bool c10::detail::integer_iterator<I, one_sided, <unnamed>>::operator!=(const c10::detail::integer_iterator<I, one_sided, <unnamed>> &) const [with I=std::size_t, one_sided=true, <unnamed>=0]"
  /home/ana/.local/lib/python3.10/site-packages/torch/include/ATen/core/qualified_name.h(73): here

  /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
    435 |         function(_Functor&& __f)
        |                                                                                                                                                 ^
  /usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
  /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
    530 |         operator=(_Functor&& __f)
        |                                                                                                                                                  ^
  /usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
  ninja: build stopped: subcommand failed.
  Traceback (most recent call last):
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1893, in _run_ninja_build
      subprocess.run(
    File "/usr/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/home/ana/text-generation-webui-main/repositories/alpaca_lora_4bit/setup.py", line 18, in <module>
      setup(
    File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.10/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.10/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run
      return orig.install.run(self)
    File "/usr/lib/python3.10/distutils/command/install.py", line 619, in run
      self.run_command('build')
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.10/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.10/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.10/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 79, in run
      _build_ext.run(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 340, in run
      self.build_extensions()
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
      build_ext.build_extensions(self)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "/usr/lib/python3/dist-packages/setuptools/command/build_ext.py", line 202, in build_extension
      _build_ext.build_extension(self, ext)
    File "/usr/lib/python3.10/distutils/command/build_ext.py", line 529, in build_extension
      objects = self.compiler.compile(sources,
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 658, in unix_wrap_ninja_compile
      _write_ninja_file_and_compile_objects(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1574, in _write_ninja_file_and_compile_objects
      _run_ninja_build(
    File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build
      raise RuntimeError(message) from e
  RuntimeError: Error compiling objects for extension
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: legacy-install-failure

× Encountered error while trying to install package. ╰─> alpaca-lora-4bit

note: This is an issue with the package mentioned above, not pip. hint: See above for output from the failure. `

caterpillarpants commented 8 months ago

I guess I cannot find a .log file. so I hope the above is useful. If there is a specific log file somewhere please tell me where it is and I can post it for you.

I have to admit. I'm kinda giddy getting tech support from a guy who made 4bit training possible for llm's. you are pretty awesome, sir.

johnsmith0031 commented 8 months ago

I found the similar issue here... https://github.com/NVIDIA/nccl/issues/650 Maybe the cause is the compatibility of cuda and std c++

caterpillarpants commented 8 months ago

Well, I am running gcc 12. I installed gcc 10 via sudo apt install gcc-10, but I don't think it's default or active. sudo update-alternatives isn't working initially. Frankly, I'm a bit over my head. I have to go to bed soon, so I'll have to take this up tomorrow. I hope to try the bisqwit solution. So update tomorrow!

caterpillarpants commented 8 months ago

well, I have nothing to update. I am now on gcc 10 and it's not working. I also have nvcc in the path, but it still can't find it.

here is my current error: ` error: [Errno 2] No such file or directory: '/home/ana/text-generation-webui-main/installer_files/env/bin/nvcc' [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for alpaca-lora-4bit Running setup.py clean for alpaca-lora-4bit Failed to build alpaca-lora-4bit ERROR: Could not build wheels for alpaca-lora-4bit, which is required to install pyproject.toml-based projects `

I had a similar problem trying to deep deepspeed to compile. My nvcc was at /usr/bin and pip or alpacalora wasn't looking there, so I put it in the path with : export PATH=/usr/bin:$PATH this fixed it for deepspeed, but isn't working here.

johnsmith0031 commented 8 months ago

try putting this after the activation of virtual env? The virtual env would overwrite the environment variables.

caterpillarpants commented 8 months ago

OK, so I've solved this nvcc issue, but have a new error. I am using text-generation-webui which has a new cmd_linux.sh command which starts a terminal and the I thought we were to do our text-gen terminal stuff with this.I assumed it started the environment, but really I have no idea. Anyway...

If I compile while using the cmd_linus.sh shell, I get the nvcc error.

If I compile without the cmd_linux.sh.shell, that it to say, in the normal terminal, I do not get the nvcc error. Instead I get these errors: Building wheel for alpaca-lora-4bit (setup.py) ... error

File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension

File "/home/ana/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 1909, in _run_ninja_build raise RuntimeError(message) from e RuntimeError: Error compiling objects for extension

There is additional output, but I didn't want to clutter the post. if you want a verbose output I can provide it.