search

artyom-beilis / pytorch_dlprim

DLPrimitives/OpenCL out of tree backend for pytorch
http://blog.dlprimitives.org/
MIT License
227 stars 16 forks source link

Request to support more functions! #54

Open CCMKCCMK opened 5 months ago

CCMKCCMK commented 5 months ago 
D:\MiniCPM\venv\Lib\site-packages\torch\_tensor.py:962: UserWarning: The operator 'aten::pow.Scalar_out' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at D:\MiniCPM\pytorch_dlprim\src\tensor_ops.cpp:313.)
  return torch.pow(other, self)
D:\MiniCPM\venv\Lib\site-packages\torch\_tensor.py:962: UserWarning: The operator 'aten::pow.Scalar_out' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at D:\MiniCPM\pytorch_dlprim\src\tensor_ops.cpp:313.)
  return torch.pow(other, self)
C:\Users\xxxxxx\.cache\huggingface\modules\transformers_modules\openbmb\MiniCPM-2B-sft-int4\xxxxxxx\modeling_minicpm.py:146: UserWarning: The operator 'aten::arange.start_out' is not currently supported on the ocl backend. Please open an issue at for requesting support https://github.com/artyom-beilis/pytorch_dlprim/issues (Triggered internally at D:\MiniCPM\pytorch_dlprim\src\tensor_ops.cpp:313.)
  t = torch.arange(self.max_seq_len_cached, device=device, dtype=self.inv_freq.dtype)

I am currently using opencl to test for LLMs <2B on my small computer. But seems some important functions are not yet supported.

tangjinchuan commented 5 months ago

Hi Artyom, I saw this question, and I wish the following would save you more time: Pow should be piece of cake. For arange, there are some GPU source for your reference: host code: https://sourceforge.net/p/octave-ocl/code/ci/default/tree/src/ocl_array.cc#l825 along with kernels https://sourceforge.net/p/octave-ocl/code/ci/default/tree/src/ocl_array_prog.cc#l34 (The author is Matthias W. Klein, I believe he is German. We were contacted via emails. Like you, we are all busy with real world stuffs. I was active in testing and patch it to AMD/Apple Silicon platforms. https://sourceforge.net/projects/octave-ocl-extra/files/) Meanwhile, the Octave CPU version: line 631: https://docs.octave.org/doxygen/4.4/d8/d00/idx-vector_8h_source.html

Best wishes, Jinchuan