Closed Archit01 closed 6 months ago
It should fit 8GB VRAM with smallish dimensions/tile size with fp8, but I'm not sure if it all will fit 16GB RAM. Everything regarding that is already implemented.
I've been able to run it with 8gb vram and 16gb ram with the following settings.
I've been able to run it with 8gb vram and 16gb ram with the following settings.
Nice, that's very good to know!
Does fp8 works with 8gb vram and 16gb ram ? Can it be implemented with tiling and offloading of models ? No matter if it is slow if it works I kindly request for this implementation.