search

CompVis / stable-diffusion

A latent text-to-image diffusion model
https://ommer-lab.com/research/latent-diffusion-models/
Other
66.51k stars 9.97k forks source link

Why bias in Q, K, V projection of SpatialSelfAttention? #859

Open SimeonZhang opened 1 week ago

SimeonZhang commented 1 week ago

https://github.com/CompVis/stable-diffusion/blob/21f890f9da3cfbeaba8e2ac3c425ee9e998d5229/ldm/modules/attention.py#L99

As I understand, other implementations of attention except for SpatialSelfAttention in this module are set with bias=False. Why is it different?

Any explanation will be greatly appreciated.