add qk norm with pretrained model，

hpcaitech / Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

https://hpcaitech.github.io/Open-Sora/

Apache License 2.0

20.1k stars 1.91k forks source link

Open lbwang2006 opened 1 week ago

lbwang2006 commented 1 week ago

add qk norm with pretrained text to image model， the generated images failed with invalid content, the training step needs to 10000 step?

JThh commented 4 hours ago

Could @zhengzangw please take a look?

JThh commented 4 hours ago

Can @lbwang2006 also provide more information on what you meant by 'needs to 10000 step' with QK norm enabled?