Add integration/CMake for SYCL backend

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

4.66k stars 334 forks source link

Add integration/CMake for SYCL backend #656

Open Jacoby1218 opened 6 months ago

Jacoby1218 commented 6 months ago

At this point, given the >4GB allocation bug was fixed, if possible, I'd like to see the SYCL backend integrated into koboldcpp. While Intel GPUs are supported by Vulkan, SYCL is much faster at prompt processing (4-7x in some cases, though Vulkan is getting faster.) Even if it's not included by default with koboldcpp (much like hipBLAS) it should be supported as a backend.

LostRuins commented 6 months ago

Unfortunately I have no way to test it as I have not gotten SYCL to run on my device.

Jacoby1218 commented 6 months ago

once #5357 is done upstream it should at least build to test on non-Intel hardware, but understandable.