Closed wilczeu closed 11 months ago
are you trying to load the model on a cpu?
Your GPU is not supported / detected so you need to do the ram requirements times 8. So in order to run the Pygmalion 6B option with Huggingface you need around 64GB of ram in your PC.
Obviously that is not what you want so the first question is if you have a suitable high memory Nvidia GPU, something 12GB and up. If the answer is no then Koboldcpp is a better solution for you.
I recommend against Pygmalion 6B since it is very old and will instead link you the much better Pygmalion 2 7B for Koboldcpp which you can run on 8GB of ram.
You can also make use of the KoboldAI Horde, or run different models on our Colab (Pygmalion got banned by Colab)