torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 512.00 MiB. GPU 0 has a total capacity of 1.96 GiB of which 225.06 MiB is free.

Leon-Sander / Local-Multimodal-AI-Chat

GNU General Public License v3.0

136 stars 83 forks source link

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 512.00 MiB. GPU 0 has a total capacity of 1.96 GiB of which 225.06 MiB is free. #24

Closed joaquindev23 closed 1 month ago

joaquindev23 commented 7 months ago

How can I solve this error?

Captura desde 2024-04-18 20-52-49

Leon-Sander commented 7 months ago

Either by using a GPU with more memory, since the message tells you that your GPU does not have enough memory. Or set gpu_layers to 0 in the config file, then the models will run on CPU.

joaquindev23 commented 7 months ago

Can you guide me to know which file I have to modify and which lines of code? Please thanks

Leon-Sander commented 7 months ago

In the config.yaml file, and if you didn't change it then line 11