Closed jhetuts closed 1 month ago
Hi @jhetuts thanks for providing feedback! Can you give a specific example on what quantized llm you are referring to? For llama3.1 we have a few quantization options and you can refer to this readme. m1 16gb should be able to run llama3.1 8B model with quantization.
Got it @larryliu0820 , I've been testing with this. Thanks!
🚀 The feature, motivation and pitch
I believe, this is what ollama's one huge advantage. This can also encourage devs to go test llm which they can run on their machine's capabilities. Like me, I have my m1 16gb, I cannot really enjoy testing the meta llms, especially the llama3.1
Alternatives
No response
Additional context
No response
RFC (Optional)
No response