Hi all!
I tried to replicate the inference times as described in this simple plot.. After multiple tries, I could reach the same performance (approx. 10 seconds, 20 + 10 steps) using the following config:
A100 GPU
small-big version (small stage B model, large stage C model)
compile = True
bs = 4
bfloat16
However, when i apply the same config to the SDXL model, I achieved a inference speed of 16.1 seconds in average, which is much more faster than stated in the histogram. Can you give some details how you compared the inference speed for your implementation?
Hi all! I tried to replicate the inference times as described in this simple plot.. After multiple tries, I could reach the same performance (approx. 10 seconds, 20 + 10 steps) using the following config:
However, when i apply the same config to the SDXL model, I achieved a inference speed of 16.1 seconds in average, which is much more faster than stated in the histogram. Can you give some details how you compared the inference speed for your implementation?