Fix SeqParallelMultiHeadCrossAttention for consistent results in distributed mode

hpcaitech / Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

https://hpcaitech.github.io/Open-Sora/

Apache License 2.0

21.76k stars 2.1k forks source link

Closed Kipsora closed 3 months ago

Kipsora commented 3 months ago

As discussed in PR #492, I propose the corresponding challenges to fix inconsistent results in distributed mode.

FrankLeeeee commented 3 months ago

Awesome, thanks!