pytorch-labs / attention-gym

Helpful tools and examples for working with flex-attention
BSD 3-Clause "New" or "Revised" License
484 stars 24 forks source link

Add Dilated Sliding Window mask_mod #12

Closed sangyeon-k closed 2 days ago

sangyeon-k commented 3 months ago

Summary

Visualization

sangyeon-k commented 3 months ago

Hi @drisspg, thanks for the review! I left a follow-up question for clarification. After that, I will update this PR!

drisspg commented 2 days ago

Landed here: https://github.com/pytorch-labs/attention-gym/pull/85