Open kristaller486 opened 2 months ago
Hello! Sorry for the late answer. It should fit .I'm planning to get to this soon. I'll keep you updated in this thread, but it may take weeks.
Thank you for answer. I'll be waiting!
Will you add Gemma2 27B as well?
Gemma2 is incredible model. With AQLM, it will fit into 12GB GPU.