vladmandic
commented
7 months ago - any first run after params change (especially lora) is slower as pipeline needs to "warm up". i'd be more interested in performance of second run with same params
- lora does impact generate performance for sure - generate now needs to apply lora weights in runtime as needed ("forward" calls). your numbers do seem extreme, but again, not unexpected.
- only way for lora NOT to have an impact would be to merge lora into model during lora load (then there would be no need for forward-apply during generate), but that would mean that every generate would have merge overhead and then unmerge overhead again at the end of the generate. so time saving would not be that great and would also make entire process less flexible.
- like you said, this impacts both diffusers and original backend (and also a1111), so not sure what is expected here under "issue"?
devils-shadow
Issue Description
When using loras in prompts, generation speeds drop proportionately to the number of loras loaded into the prompt. This behavior is replicated consistently across both diffusers and original backends and is sampler agnostic.
Example: sdnext with diffusers, dreamshaper 1.5, all default settings, default extensions, default generation parameters, 50steps: 19it/s. Loading one lora into the prompt drops gen speed to about 11it/s and the speed drop continues as more loras are added.
https://github.com/vladmandic/automatic/assets/17099756/b4dfa5d7-1ce6-4a25-b154-65242ad6cf14
Version Platform Description
SDNext Version: app=sd.next updated=2023-12-04 hash=93f35ccf Python version 3.10.11 Windows 11 23H2 OS build 22631.2792 Nvidia 4070 Driver Version 546.29
Relevant log output
Backend
Diffusers
Branch
Master
Model
SD 1.5
Acknowledgements