These are helpers and scripts for using Intel Arc gpus with oobabooga's text-generation-webui. Contributions welcome.
See the original post for more details.
This is just a starting point. Changes are welcome. Once it's right in bash, we can decide whether to integrate it with oobabooga's start_linux.sh, requirements files, and one_click.py.
Download or clone a fresh copy of Oobabooga.
Save the below scripts into text-generation-webui
. These should be in the same folder as one_click.py
, cmd_linux.sh
, etc.
Make them executable.
cd text-generation-webui
./install_arch.sh
Check clinfo
for your hardware information.
clinfo -l
In run_arc.sh
, find GGML_OPENCL_PLATFORM
and change it to your platform number. Then change the GGML_OPENCL_DEVICE
to your device name. If you have not added OneAPI's setvars.sh to your .bashrc, uncomment the source line. Save the file.
Start the server with run_arch.sh
. This uses any flags you've saved in CMD_FLAGS.txt
. You can also use flags like --listen --api
with the script.
./run_arch.sh