Img AI

[parsehex]

This is image-generating AI, initially supporting Stable Diffusion and then probably DALL-E 3 afterwards.

To use SD models I'll be using the Diffusers library.

• [x] Txt2Img
• [ ] Img2Img
• [ ] Inpainting
• [ ] Outpainting
• [x] SD (non-XL)
• [x] SDXL
• [ ] SD(XL) Turbo
• [x] Getting / Listing models
• [ ] (Multiple) Samplers - Support all the good new samplers
  • It seems that they're called Schedulers in the Diffusers lib. See here for a table listing (some?) available schedulers and their corresponding name in Automatic1111. So, we'll probably make a map of schedulers and pass the keys as the sampler list.
  • [x] Listing samplers
  • [x] Using samplers
• [x] Support changing Seed
• [ ] Support Textual Inversions (e.g. easynegative)
• [ ] Support LoRAs
• [ ] StableDiffusionCpp client https://github.com/leejet/stable-diffusion.cpp
• [x] Return the used generation parameters
  • [ ] And infotext
• [ ] Support batching
• [ ] Prompt weights (example)

[parsehex]

Basic SDXL support, haven't tried the refiner but results from base model alone were disappointing, reasonable speed.

Note dump

• Looks like you change the "scheduler" by setting the pipeline.scheduler reference to the desired sampler (e.g. pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config)) - See here for a table of different schedulers
• Info on using Textual Inversion. Also, this page covers Adapters (includes text inv. and LoRAs)
• This page mentions using a Seed, also shows batching
• SDXL Turbo article
• img2img article, and inpainting

[parsehex]

Trying to switch sampler/scheduler... Why is it this kind of difficult? It seems to be that the only way to change the thing is to re-load the model with a different sampler. How in the the world do all of these webuis let you hot-swap the damn sampler???

I'm skeptical that changing the pipeline.scheduler even works despite it being all over the docs (here). See last comment here -- someone saying that setting the pipeline.scheduler can lead to errors and not to encourage it. And yet this is what the docs say to do.

[parsehex]

Okay, I think I got there with changing the scheduler. The main thing:

Re-create the AutoPipelineForText2Image after changing the scheduler (so set self.model)

Also still setting default_config to pipeline.scheduler.config before initially setting the default sampler. Then, use this default config to initialize later samplers (should we cache samplers to avoid re-making?)