[FEATURE] Add SYCL feature flags to rllm-llamacpp build (To add support for Intel GPUs) - Githubissues

microsoft / aici

AICI: Prompts as (Wasm) Programs

MIT License

1.87k stars 76 forks source link

[FEATURE] Add SYCL feature flags to rllm-llamacpp build (To add support for Intel GPUs) #95

Closed TheMrCodes closed 3 months ago

TheMrCodes commented 3 months ago

The goal of this pull request is to add support for Intel GPU to the build script of rllm llama.cpp This is done by

expanding the cargo feature flags of rllm-llamacpp and rllm-llamacpp_low and incoporate
adding the needed build arguments in the main.rs from rllm-llamacpp_low
and adding appropriate new options to server.sh

The code will be tested on a machine with Intel 13Gen Processor & Intel Arc A770 GPU