[Bug]: Prompt editing schedule behaving strangely in certain samplers

[dothere]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

Schedules for prompt editing appear to not be broken/not behaving as intended on some samplers, the ones I've noticed the issue have been DPM++ 2Sa, DPM++ SDE, DPM++ 2Sa Karras, and DPM++ SDE Karras.

It looks like it is completing the schedule in half the amount of steps and then reverting to the original prompt.

Steps to reproduce the problem

Select either DPM++ 2Sa, DPM++ SDE, DPM++ 2Sa Karras, or DPM++ SDE Karras samplers and generate the following prompt:

[dog:cat:0.5] at 100 sampling steps (CFG scale, width/height, other params don't seem to make a difference)

will generate dog for 25 steps, cat for another 25 steps, and then generate dog for the last 50 steps.

Even if the number of steps is specified (i.e. [dog:cat:50]) it will still change to cat at 25 and then change back to dog at 50.

It should be noted that this is also occurring in the hires fix pass.

What should have happened?

Normal behavior for the prompt [dog:cat:0.5] at 100 steps should be Generate "dog" for 50 steps Generate "cat" for 50 steps

When it passes through hires fix, it should follow this schedule as well.

Commit where the problem happens

226d840e84c5f306350b0681945989b86760e616

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

None as I have rebuilt webui to ensure I could reproduce the issue (it does appear to be reproducible on this repo)

My own local branch where I first noticed the issue has:

--xformers --autolaunch --disable-safe-unpickle --no-half-vae --opt-sub-quad-attention

List of extensions

None, did a reinstall of webui to ensure a clean build

Console logs

venv "P:\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Commit hash: 226d840e84c5f306350b0681945989b86760e616
Installing requirements for Web UI
Launching Web UI with arguments:
No module 'xformers'. Proceeding without it.
Calculating sha256 for P:\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.ckpt: cc6cb27103417325ff94f52b7a5d2dde45a7515b25c255d8e396c90014281516
Loading weights [cc6cb27103] from P:\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.ckpt
Creating model from config: P:\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(0):
Model loaded in 16.4s (calculate hash: 10.9s, load weights from disk: 1.5s, create model: 0.6s, apply weights to model: 0.6s, apply half(): 0.8s, move model to device: 1.0s, load textual inversion embeddings: 1.0s).
Checkpoint v1-5-pruned-emaonly.ckpt [cc6cb27103] not found; loading fallback v1-5-pruned-emaonly.ckpt [cc6cb27103]
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
100%|████████████████████████████████████████████████████████████████████████████████| 100/100 [00:18<00:00,  5.29it/s]
100%|████████████████████████████████████████████████████████████████████████████████| 100/100 [01:45<00:00,  1.06s/it]
Total progress: 100%|████████████████████████████████████████████████████████████████| 200/200 [02:10<00:00,  1.53it/s]
{"prompt": "[dog:cat:0.5]", "all_prompts": ["[dog:cat:0.5]"], "negative_prompt": "", "all_negative_prompts": [""], "seed": 4257038347, "all_seeds": [4257038347], "subseed": 1239457695, "all_subseeds": [1239457695], "subseed_strength": 0, "width": 512, "height": 512, "sampler_name": "DPM++ 2S a", "cfg_scale": 7, "steps": 100, "batch_size": 1, "restore_faces": false, "face_restoration_model": null, "sd_model_hash": "cc6cb27103", "seed_resize_from_w": 0, "seed_resize_from_h": 0, "denoising_strength": 0.94, "extra_generation_params": {"Hires upscale": 2, "Hires upscaler": "Latent"}, "index_of_first_image": 0, "infotexts": ["[dog:cat:0.5]\nSteps: 100, Sampler: DPM++ 2S a, CFG scale: 7, Seed: 4257038347, Size: 512x512, Model hash: cc6cb27103, Model: v1-5-pruned-emaonly, Denoising strength: 0.94, Hires upscale: 2, Hires upscaler: Latent"], "styles": [], "job_timestamp": "20230203172830", "clip_skip": 1, "is_using_inpainting_conditioning": false}

Additional information

No response

[TimothyKrell]

I'm having this problem as well. I've verified that on DPM++ SDE Karras it indeed switches with [aloy from horizon zero dawn:henry cavil superman:0.5] it uses the first part for 25% of the time, then the second prompt for 25% of the time, then reverts to the first prompt.

It does not do this on DPM++ 2M Karras and Euler A.

[baobabKoodaa]

I can also confirm this issue exists for me in at least DPM++ SDE Karras and I can also confirm it does not occur on DPM++ 2M Karras.

[bbonvi]

Experiencing the same issue. Have you found a workaround?

[catboxanon]

This is because the model invokes those samplers twice per step (something those samplers do by design), so the syntax is basically ran twice as fast. I'll try looking into this further if I can but this note could maybe help someone else out.