Open chengzeyi opened 1 month ago
cc: @bertmaher , I'm guessing this is using the new triton 3.0 branch? Not sure how to tell which triton hash is being used there.
cc: @bertmaher , I'm guessing this is using the new triton 3.0 branch? Not sure how to tell which triton hash is being used there.
Thank you for your quick response. This should be the upcoming 3.0 version of triton. The commit hash is in the packaged wheel file name.
Another hint is that if I use fp32 dot accumulation instead of fp16, the output will be correct.
Hello, guys. Thank you for all your great work on this awesome project! I am currently building a new deep learning acceleration framework with it. But I have some problems with it now. Hope you could give me a help.
I am currently working with
nightly torch
, which is installed by:This version of
pytorch
ships with a quite new version oftrion
, that is:This
triton
can be installed withpip3 install --pre -U pytorch_triton==3.0.0+dedb7bdf33 --index-url https://download.pytorch.org/whl/nightly/cu124
Ok, then I have a quite complex fused attention kernel. In fact, it is written with
torchinductor
as aTritonTemplate
, and is expected to have a very impressive performance. Now I have extracted the code the generated fromtorch_compile_debug
folder. I also add a test assert to it to check if the output is correct.After running it, The console prints that the assertion failed and the output tensor contains
inf
values. If I usetl.device_print
to see the values of the finalacc_0
beforetl.store
, it seems that the values are Ok, which is quite strange.However, if I switch to an older
triton
, for example:This version of
triton
can be installed bypip3 install --pre -U pytorch_triton==3.0.0+a9bc1a3647 --index-url https://download.pytorch.org/whl/nightly/cu124
. It's not very old though.With this older version of
triton
. The output tensor does not containinf
and is relatively correct, and does not trigger the assertion failure.Currently I have no idea about why this happens. I use an
RTX 4090
card now. This is howtorch.utils.collect_env
shows. Do you have any idea?