search

mcmonkeyprojects / sd-infinity-grid-generator-script

Infinite-Axis Grid Generator for Stable Diffusion!
MIT License
182 stars 25 forks source link

Batching #114

Closed yggdrasil75 closed 1 year ago

yggdrasil75 commented 1 year ago

I will fix variable names not being snake_case and test before marking ready.

yggdrasil75 commented 1 year ago

The batched started at: 2023-10-27 21:49:23.310027 The batched ended at: 2023-10-27 22:05:54.651681

The unbatched started at: 2023-10-27 22:07:15.212591 The unbatched ended at: 2023-10-27 22:27:06.748224

batched: ~16 minutes. unbatched: ~20 minutes. ~20% benefit. (this is with the short example, which doesnt reach peak due to unmergeable changes. on a 3090, peak for batch overall 512x on sd 1.5 is ~30, not 12. this does change with samplers, model type, and resolution.

note on unmergeable changes: seed, prompt, negative prompt, and subseed are considered "mergeable", any other differences are not. model changes will immediately break a group in 2, cfg scale and steps will as well. there are too many other unmergeables to immediately break all groups, so a validator confirms the groups.

this does touch a lot of things in the origin, I kept trying to get it with minimal changes, and trying to resolve as many issues as I could, it is slightly better than what is currently in my fork, but most of the performance improvements got scrapped due to various things breaking. I may reimplement them later, but for now its just slightly better instead of a lot better.

mcmonkey4eva commented 1 year ago

You'd need a whitelist of batchables to prevent errors - only a few specific options are batchable, all the rest are explicitly unbatchable - ie they must be equal to permit batching. You should not have blacklists on model/steps/cfg, that is a tiny subset of all the batch-breaking options.