Add matrix bias to forward/backward kernel

ROCm / aotriton

Ahead of Time (AOT) Triton Math Library

MIT License

42 stars 15 forks source link

Add matrix bias to forward/backward kernel #14

Closed xinyazhang closed 7 months ago

xinyazhang commented 7 months ago

All UTs Passed. No performance regression on existing forward kernel cases (i.e., test/performance_forward.py).

xinyazhang commented 7 months ago

I'm trusting the database changes but I was also concerned to not see testing above 2k sequence lengths as most use cases where we are beneficial are higher than that.

Any sequence longer than 2k will be treated as 2k when looking up the optimal kernel. I don't think the optimal compiler options will change for longer sequences.