Closed feynmanish closed 8 months ago
Hi feynmanish, can you share which HuggingFace model are you trying to load? Also, please share your GPU VRAM. HuggingFace models require high VRAM as they run locally.
128 mb, but there is a similar issue with most capable machines: https://github.com/invoke-ai/InvokeAI/issues/2234
@Abdullahw72 : Great work, But wanted to know what should be bare minimum system requirement to run the code + for every user we need to have separate instance to chat. How much extra memory it will consume
@PinkalShah2022 Thank you! It really depends on which model you want to use, to be honest. If you use a hugging face model, you'll need to look for really small models like Falcon 7B parameters, it will require at least 12 GB VRAM and a pretty decent CPU. Look for specs of each model on Huggingface under the model descriptions! Same for the memory, depends on use-case but 16 -32 GB RAM would be a good place to begin for any small/medium sized LLM.
It works with openAi, but with HuggingFace Getting "Unable to load weights from pytorch checkpoint file" when loading model from transformers error occurs. Any suggestions?