Closed sayakpaul closed 1 month ago
@dacorvo a gentle ping :)
I reproduce the issue. Somewhere in the third transformer block nan
outputs are generated. I suspect this comes from one the quantized weights after dequantization (and not from a torch.float16
operation). I am checking them one by one to find out which one.
It works well with int8 though ...
Somewhere in the third transformer block nan outputs are generated. I suspect this comes from one the quantized weights after dequantization (and not from a torch.float16 operation) So I was wrong: this was indeed an overflow in a
torch.float16
matmul, but only because I applied thefloat8
quantization scale after the matmul. By applying the scale to thefloat8
weights before the matmul they are back in the numerical range they had before quantization and the overflow disappears.
Here are the images generated with a float16 transformer and a float8 transformer (very subtle differences in the towers and the landscape is a bit different).
I am pushing the fix with a working example. I also quantized the text_encoder to make a real difference in size.
I am on the
main
of quanto.leads to a black image