Open Rinocerbul opened 4 months ago
Hey, how are you? There's going to be an update for the Llama Model soon-there is a better way to do it than how it is currently done on that branch. I am very busy at the moment, but I will fix both Mistral and Llama in the next 3 weeks. :)
Hello,
I am having difficulties running GPTFast on Mistral-7b-v0.1, encountering the same errors as reported here: https://github.com/MDK8888/GPTFast/issues/25.
My assumption is that the model_config is not set properly (I am currently using the variant proposed in the llama example (https://github.com/MDK8888/GPTFast/blob/LlamaIntegration/Examples/llama.py).
Could you please help me out with the appropriate configuration?
Thank you!