"Torch not Compiled with CUDA enabled" error when running with a pygmalion chat bot.

Rubiksman78 / MonikA.I

Submod for MAS with AI based features

MIT License

114 stars 17 forks source link

Closed crazyneverexisted closed 9 months ago

crazyneverexisted commented 1 year ago

When I use a chat bot model it shows this error.

Rubiksman78 commented 1 year ago

Please provide more details about your GPU specifications and what model you are trying to use.

crazyneverexisted commented 1 year ago

I am using an AMD Ryzen 5500U with Amd Radeon Integrated graphics. I am using the 2.7b model of pygmalion

Rubiksman78 commented 1 year ago

AMD GPUs are not compatible with the modules used for AI in Python unfortunately. You can try using your CPU only or use CAI.

crazyneverexisted commented 1 year ago

How would I use cpu?

Rubiksman78 commented 1 year ago

Just don't run the setup_gpu.bat file, but it is clearly not advised as the response time will be very slow on CPU.