search

henk717 / KoboldAI

KoboldAI is generative AI software optimized for fictional use, but capable of much more!
http://koboldai.com
GNU Affero General Public License v3.0
359 stars 130 forks source link

problem with loading Pygmalion-6B #469

Closed wilczeu closed 11 months ago

wilczeu commented 11 months ago

kobold-error image

RecoveredApparatus commented 11 months ago

are you trying to load the model on a cpu?

henk717 commented 11 months ago

Your GPU is not supported / detected so you need to do the ram requirements times 8. So in order to run the Pygmalion 6B option with Huggingface you need around 64GB of ram in your PC.

Obviously that is not what you want so the first question is if you have a suitable high memory Nvidia GPU, something 12GB and up. If the answer is no then Koboldcpp is a better solution for you.

I recommend against Pygmalion 6B since it is very old and will instead link you the much better Pygmalion 2 7B for Koboldcpp which you can run on 8GB of ram.

You can also make use of the KoboldAI Horde, or run different models on our Colab (Pygmalion got banned by Colab)