Closed Manni1000 closed 1 month ago
Thank you for your interest! Releasing a quantized model is on our TODO list!
You probably need a GPU with compute capability >= 8.0 (for bf16 ops) and VRAM > 32GB.
how much vram do i need for fine tuning or inference?
Just tried it and saw it used 28Gb VRAM.
thanks
how much vram do i need for fine tuning or inference?