Closed ZFTurbo closed 4 months ago
I noticed error in Linear Attention layer when flash_attn is set in True:
flash_attn
True
File "attend.py", line 84, in flash_attn out = F.scaled_dot_product_attention( RuntimeError: No available kernel. Aborting execution.
In standard Attention (time_transformer and freq_transformer) it works ok.
time_transformer
freq_transformer
So currently as workaround I did followig change in LinearAttention (set False to flash):
LinearAttention
self.attend = Attend( scale=scale, dropout=dropout, flash=False )
@ZFTurbo ohh ok, i've made it configurable using linear_flash_attn kwarg
linear_flash_attn
I noticed error in Linear Attention layer when
flash_attn
is set inTrue
:In standard Attention (
time_transformer
andfreq_transformer
) it works ok.So currently as workaround I did followig change in
LinearAttention
(set False to flash):