search

black-forest-labs / flux

Official inference repo for FLUX.1 models
Apache License 2.0
15.34k stars 1.1k forks source link

What GPU card is officially recommended for model inference in Flux.1 [Dev]? #176

Open keymonzhang opened 1 week ago

keymonzhang commented 1 week ago

I hope someone can help me answer it. Thanks!

johnwick123f commented 1 week ago

@keymonzhang Nothing really officially recommended. Powerful gpu's like h100 should be the fastest right now especially with fp8, however most people don't have access to them. A 4090 with the gguf q8 version will be decently fast and perform nicely.

The lowest you can run it comfortably is with a good 8gb vram gpu and a nf4 quantized version. It will be slightly worse then the original one(bf16) but can fit in 8gb vram unlike bf16 which requires at least 24gb at the least.