search

v0xie / sd-webui-incantations

Enhance Stable Diffusion image quality, prompt following, and more through multiple implementations of novel algorithms for Automatic1111 WebUI.
GNU General Public License v3.0
130 stars 8 forks source link

Great promise; new toy! Hard Prompts Made Easy #3

Open tau0-deltav opened 7 months ago

tau0-deltav commented 7 months ago

v0xie it's only now I'm noticing that a lot of my favourite extensions from A1111 have been implemented, contributed to or even outright designed by you, so seeing them all come together with other good ideas is very exciting. Thank you and good luck to you and your co-conspirators.

I submit a neglected but (I think?) relevant method for your attention:

Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery https://doi.org/10.48550/arXiv.2302.03668

Exists as an unlisted (heartbreaking!) extension for A1111 as the mighty Pez Dispenser. You will also find a link to the reference code in both the paper above and the extension here: https://github.com/r0mar0ma/sd-webui-pez-dispenser

I'm submitting this just assuming you'd be interested to see it. As I don't use A1111 but SD.Next instead, (where fearsome Pez Dispenser yet lives), I'm not really invested in this repo beyond just seeing good work come together.

I do miss CADS a great deal though... that algorithm is genuine magic. Are diffusers implementations of your own methods plausible (i.e matters of semantics and not design philosophy)? A1111 just does so many silly silly things that I just couldn't cope with its inconsistency any more. But work like yours is a serious loss for those making the switch. Don't you deserve torch.compile(), v0xie? Don't we all?

v0xie commented 7 months ago

Thank you for the kind words, I'm happy people find my extensions useful. I can't take credit for any of the algorithms, created by people much smarter than me... lol.

It's funny you should mention PEZ since yesterday I implemented (read: copied) it from the official code repository. It seems like a good framework to build upon for prompt optimization and learning.

Regarding diffusers, at least one of the methods are already part of diffusers.

I would think many A1111 Extensions would be compatible with SD.Next since it's built on top of A1111?

Please feel free to send me any interesting papers / methods you find, I'm currently scouring the depths of ArXiv for neglected methods to attempt to implement.

StableInfo commented 6 months ago

Please feel free to send me any interesting papers / methods you find, I'm currently scouring the depths of ArXiv for neglected methods to attempt to implement.

I think you can apply to be in the same category as the smarter people you mentioned this is really cool. I have never ever used anything mentioned here except PEZ (hard prompts..) I found this by searching 'hard prompt.." in the search bar. I will try this extention later, any guide made by prople on how to use it etc? I mean Its first time I heard about CADS etc..