search

oobabooga / one-click-installers

Simplified installers for oobabooga/text-generation-webui.
GNU Affero General Public License v3.0
550 stars 186 forks source link

compiling quant_cuda_kernel.o failed #33

Closed jcl2023 closed 1 year ago

jcl2023 commented 1 year ago

This is the error I got:

/home/test/2TB/GITS/text-generation-webui/one-click-installers/text-generation-webui/repositories/GPTQ-for-LLaMa/quant_cuda_kernel.cu(654): error: identifier "__hfma2" is undefined

Any idea why?

jllllll commented 1 year ago

The cuda GPTQ-for-LLaMa used by the webui is from oobabooga's fork which does not support GPUs older than Pascal. The latest cuda kernels from GPTQ support late-Kepler+ (compute 3.5+) and should work on your system. Though, you won't be able to run models quantized with ooba's fork with that.

Eventually, we should move to something like 0cc4m's fork for broader GPU and model support.

github-actions[bot] commented 1 year ago

This issue has been closed due to inactivity for 30 days. If you believe it is still relevant, please leave a comment below.