search

IST-DASLab / gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".
https://arxiv.org/abs/2210.17323
Apache License 2.0
1.81k stars 145 forks source link

OpenCL Support #20

Closed apcameron closed 1 year ago

apcameron commented 1 year ago

Please add OpenCL Support that so that it can be used on GPU's that Support OpenCL and not CUDA Then we could use something like quant_opencl.cpp instead of quant_cuda.cpp

efrantar commented 1 year ago

Unfortunately, we have currently no plans for adding non-CUDA GPU kernels. However, I suspect that, in principle, similar techniques as the ones we are using for our CUDA kernels probably also work in other GPU / accelerator programming frameworks, hence quant_cuda.cpp may be a helpful (higher-level) reference for such implementations.