search

pytorch / pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration
https://pytorch.org
Other
84.45k stars 22.74k forks source link

CuDNN SDPA Issue Tracker #141133

Open drisspg opened 5 days ago

drisspg commented 5 days ago

Summary

Keeps track of known issues and the current state of the integration, with the goal to bump order in priority list with higher confidence

### Tasks
- [ ] https://github.com/pytorch/pytorch/issues/139298
- [x] Match Query's stride layout https://github.com/pytorch/pytorch/pull/138354
- [ ] gradOutput vs Ouput stride mismatch, blocked by cudnn bump: https://github.com/pytorch/pytorch/pull/138354#issuecomment-2488863424
- [ ] Fill Value twiddiling: https://github.com/pytorch/pytorch/pull/140837
- [ ] Debugging PR: https://github.com/pytorch/pytorch/pull/139699
- [ ] Update CuDNN backend to respect enable_gqa https://github.com/pytorch/pytorch/issues/139586
- [ ] Mitigated: https://github.com/pytorch/pytorch/issues/138529 but should see if fixed in new version of cudnn
- [ ] WSL: https://github.com/huggingface/diffusers/issues/9704 | need to fix or guard correctly

cc @csarofeen @ptrblck @xwang233 @msaroufim @mikaylagawarecki

Skylion007 commented 5 days ago

Tossing this PR into the list: https://github.com/pytorch/pytorch/pull/141178