search

ironjr / StreamMultiDiffusion

Official code for the paper "StreamMultiDiffusion: Real-Time Interactive Generation with Region-Based Semantic Control."
https://jaerinlee.com/research/streammultidiffusion
MIT License
518 stars 43 forks source link

stablemultidiffusion_pipeline.py code is incomplete about the code that does not apply to the lcm method. Can you provide a copy? #11

Open redredbluee opened 3 months ago

redredbluee commented 3 months ago

stablemultidiffusion_pipeline.py code is incomplete about the code that does not apply to the lcm method. Can you provide a copy? there is something wrong with the scheduler

ironjr commented 3 months ago

Ah, yes. I do have a plan to support other types of acceleration LoRAs (I'm trying Hyper-SD right now).

If you have a list of request for other types of schedulers, please let me know. Thanks!

redredbluee commented 3 months ago

DDIMScheduler

Ah, yes. I do have a plan to support other types of acceleration LoRAs (I'm trying Hyper-SD right now).

If you have a list of request for other types of schedulers, please let me know. Thanks!

Ah, yes. I do have a plan to support other types of acceleration LoRAs (I'm trying Hyper-SD right now).

If you have a list of request for other types of schedulers, please let me know. Thanks!

not using any acceleration, I want to use the normal pipeline with DDIMScheduler, 20-50 step. But the code currently does not support. Does this algorithm require an acceleration solution?

ironjr commented 3 months ago

Ah the algorithm is originally about acceleration stuff, but I will include other standardized samplers as well. Thanks for the feedback. The update won't take so long.

redredbluee commented 3 months ago

Ah the algorithm is originally about acceleration stuff, but I will include other standardized samplers as well. Thanks for the feedback. The update won't take so long.

This algorithm is great. When the mask area is small during inpainting, I wanted a more detailed and controlled generation, while the result is a bit rough. I'm wandering whether the acceleration affects.

andupotorac commented 3 months ago

Curious if you manage to add support to LCM-LORA. I see this is already supported in the original StreamDiffusion, so shouldn't it work here as well?

ironjr commented 3 months ago

LCM-LoRA's already in support, though I have found that LCM-SDXL has lower quality and relatively severe style leakage issues compared to newer acceleration modules (Lightning/Hyper-SD/etc...). It'll be nice if you suggest more acceleration methods, so I can put them into this :) And... I am sorry for the late update of DDIMSampler. This should be my top priority.

andupotorac commented 3 months ago

Sure, here are a few accelerators:

Also check OneDiff, FaserLCM, DeepCache, if these are useful.

ironjr commented 3 months ago

Copy that!