anapnoe
commented
1 year ago Thanks for this PR I am waiting for a response from @vladmandic there is some discussion to merge our efforts ideally under an organization that will include more devs and maintainers so I don't want to introduce any PRs at the moment while for sure this one is a go for me I will wait for his response
vladmandic
Credit for this pull request goes to papuSpartan. This is almost entirely copied from his pull request.
This pull request integrates the speed and memory improvements from the paper Token Merging for Fast Stable Diffusion via dbolya/tomesd. These changes do alter reproducibility of past generations. For that reason, I have set this as an option that is disabled by default but can be tweaked through its settings pane.
All settings can be found in its section in under the settings tab.
From direct suggestion of the author, the default behavior is to not apply token merging except to the hi-res fix pass. From my testing so far, this generally results in the end image looking cleaner. Undoubtedly, this is due to hr-fix being handed a more coherent image to begin with. If the user still wants to apply token merging globally though, that option is made available in the settings pane.
Tested in Windows 11 and Linux.
Citations
Bolya, D., & Hoffman, J. (2023). Token Merging for Fast Stable Diffusion. ArXiv. Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., & Hoffman, J. (2023). Token Merging: Your ViT but Faster. International Conference on Learning Representations.