Open ElliottDyson opened 4 months ago
I would be more than happy to help, I'd just need a pointer to the areas where things are currently being loaded to Nvidia GPUs so it can be rewritten for Ipex as the file structure you have here is rather dense (there would probably have to be a separate download for Ipex tbh).
Hello, probably a big ask, but would you consider adding intel arc GPU acceleration (it also invariantly speeds up Intel CPUs), however, there are plenty of examples over in the IPEX-LLM repo for implementations. It's a long shot, but it would truly be great if possible.
All that would be required for the user would be a warning to visit the IPEX-LLM docs to install the necessary library.
The examples include loading to and from the GPU, etc., and includes a pytorch version specific for the GPUs.
Hopefully this is possible, as without GPU acceleration the program can be really rather slow.
Thank you.