[Issue]: Output images will not follow prompts (WSL + IPEX)

[cyatarow]

Issue Description

I did a clean install of SD.Next and tried generating images, but the images did not follow the prompts and make sense.

Example: Model-> YesMix v5.0 Prompt-> 1girl, forest, masterpiece, best quality Negative-> (low quality:1.3), (worst quality:1.3),(monochrome:0.8),(deformed:1.3),(malformed hands:1.4),(poorly drawn hands:1.4),(mutated fingers:1.4),(bad anatomy:1.3),(extra limbs:1.35),(poorly drawn face:1.4),(watermark:1.3),(patreon logo) Size-> 512x768 Guidance scale-> 8 Steps-> 25 Sampling method-> DPM++ 2M Seed-> 2368381021

Version Platform Description

• SD.Next: updated=2024-10-24 hash=06dd9e5f branch=master
• OS: Windows 11 Pro 23H2 + WSL Ubuntu 22.04.5
• GPU: Intel Arc A750 (IPEX)
• Browser: Firefox Portable 131.0.3

02:33:07-751138 INFO     Starting SD.Next
02:33:07-753856 INFO     Logger: file="/home/arc22/sdx/ca/sdnext.log" level=INFO size=34779 mode=append
02:33:07-755077 INFO     Python: version=3.10.12 platform=Linux bin="/home/arc22/sdx/ca/venv/bin/python3" venv="/home/arc22/sdx/ca/venv"
02:33:07-764323 INFO     Version: app=sd.next updated=2024-10-24 hash=06dd9e5f branch=master
                         url=https://github.com/vladmandic/automatic/tree/master ui=main
02:33:08-241674 INFO     Platform: arch=x86_64 cpu=x86_64 system=Linux release=5.15.153.1-microsoft-standard-WSL2 python=3.10.12
02:33:08-270613 INFO     Python: version=3.10.12 platform=Linux bin="/home/arc22/sdx/ca/venv/bin/python3" venv="/home/arc22/sdx/ca/venv"
02:33:08-271590 INFO     IPEX: Intel OneAPI toolkit detected
02:33:08-305675 INFO     Verifying requirements
02:33:08-311217 INFO     Verifying packages
02:33:08-314636 INFO     Extensions: disabled=[]
02:33:08-315564 INFO     Extensions: enabled=['sd-extension-system-info', 'Lora', 'sd-webui-agent-scheduler', 'sd-extension-chainner',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg'] extensions-builtin
02:33:08-316681 INFO     Extensions: enabled=[] extensions

Relevant log output

arc22@machine-RE2:~$ $HOME/sdx/ca/webui.sh --use-ipex
Activate python venv: /home/arc22/sdx/ca/venv
Launch: venv/bin/python3
02:33:07-751138 INFO     Starting SD.Next
02:33:07-753856 INFO     Logger: file="/home/arc22/sdx/ca/sdnext.log" level=INFO size=34779 mode=append
02:33:07-755077 INFO     Python: version=3.10.12 platform=Linux bin="/home/arc22/sdx/ca/venv/bin/python3" venv="/home/arc22/sdx/ca/venv"
02:33:07-764323 INFO     Version: app=sd.next updated=2024-10-24 hash=06dd9e5f branch=master
                         url=https://github.com/vladmandic/automatic/tree/master ui=main
02:33:08-241674 INFO     Platform: arch=x86_64 cpu=x86_64 system=Linux release=5.15.153.1-microsoft-standard-WSL2 python=3.10.12
02:33:08-270613 INFO     Python: version=3.10.12 platform=Linux bin="/home/arc22/sdx/ca/venv/bin/python3" venv="/home/arc22/sdx/ca/venv"
02:33:08-271590 INFO     IPEX: Intel OneAPI toolkit detected
02:33:08-305675 INFO     Verifying requirements
02:33:08-311217 INFO     Verifying packages
02:33:08-314636 INFO     Extensions: disabled=[]
02:33:08-315564 INFO     Extensions: enabled=['sd-extension-system-info', 'Lora', 'sd-webui-agent-scheduler', 'sd-extension-chainner',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg'] extensions-builtin
02:33:08-316681 INFO     Extensions: enabled=[] extensions
02:33:08-317431 INFO     Startup: standard
02:33:08-318010 INFO     Verifying submodules
02:33:15-181453 INFO     Extensions enabled: ['sd-extension-system-info', 'Lora', 'sd-webui-agent-scheduler', 'sd-extension-chainner',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg']
02:33:15-182393 INFO     Verifying requirements
02:33:15-184305 INFO     Command line args: ['--use-ipex'] use_ipex=True
02:33:19-132874 INFO     System packages: {'torch': '2.3.1+cxx11.abi', 'diffusers': '0.32.0.dev0', 'gradio': '3.43.2', 'transformers': '4.45.2',
                         'accelerate': '1.0.0'}
02:33:19-516091 INFO     Device detect: memory=7.9 ptimization=medvram
02:33:19-518340 INFO     Engine: backend=Backend.ORIGINAL compute=ipex device=xpu attention="Scaled-Dot-Product" mode=no_grad
02:33:19-519314 WARNING  Backend=original is in maintainance-only mode
02:33:19-821649 INFO     Torch parameters: backend=ipex device=xpu config=Auto dtype=torch.bfloat16 vae=torch.bfloat16 unet=torch.bfloat16
                         context=no_grad nohalf=False nohalfvae=False upscast=False deterministic=False test-fp16=True test-bf16=True
                         optimization="Scaled-Dot-Product"
02:33:20-101286 INFO     Device: device=Intel(R) Graphics [0x56a1] n=1 ipex=2.3.110+xpu
02:33:20-240571 INFO     Available VAEs: path="models/VAE" items=0
02:33:20-241673 INFO     Available UNets: path="models/UNET" items=0
02:33:20-242534 INFO     Available TEs: path="models/Text-encoder" items=0
02:33:20-243281 INFO     Disabled extensions: ['sdnext-modernui']
02:33:20-246715 INFO     Available Models: path="models/Stable-diffusion" items=1 time=0.00
02:33:20-274615 INFO     Available Yolo: path="models/yolo" items=6 downloaded=0
02:33:20-316736 INFO     Available LoRAs: items=0 folders=2
02:33:20-962921 INFO     Extension: script='extensions-builtin/sd-webui-agent-scheduler/scripts/task_scheduler.py' Using sqlite file:
                         extensions-builtin/sd-webui-agent-scheduler/task_scheduler.sqlite3
02:33:20-974492 INFO     Available Upscalers: items=50 downloaded=0 user=0 time=0.01 types=['None', 'Lanczos', 'Nearest', 'ChaiNNer', 'LDSR',
                         'SCUNet', 'SD', 'RealESRGAN', 'SwinIR', 'AuraSR', 'ESRGAN']
02:33:20-981205 INFO     Available Styles: folder="models/styles" items=288 time=0.00
02:33:20-983385 INFO     UI theme: type=Standard name="black-teal"
02:33:21-426465 INFO     Extension list is empty: refresh required
02:33:21-832525 INFO     Local URL: http://127.0.0.1:7860/
02:33:21-990982 INFO     [AgentScheduler] Task queue is empty
02:33:21-992049 INFO     [AgentScheduler] Registering APIs
02:33:22-078035 INFO     Model metadata saved: file="metadata.json" items=1 time=0.00
02:33:22-080411 INFO     Load model: select="CheckpointYesmix_v50 [efb352a7cb]"
Load model: /home/arc22/sdx/ca/models/Stable-diffusion/CheckpointYesmix_v50.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/2.1 GB -:--:--
tokenizer_config.json: 100%|███████████████████████████████████████████████████████████████████████████████████| 905/905 [00:00<00:00, 8.93MB/s]
vocab.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 961k/961k [00:00<00:00, 1.47MB/s]
merges.txt: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 525k/525k [00:00<00:00, 1.09MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████████████████████████████| 389/389 [00:00<00:00, 3.46MB/s]
tokenizer.json: 100%|██████████████████████████████████████████████████████████████████████████████████████| 2.22M/2.22M [00:00<00:00, 4.23MB/s]
config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 4.52k/4.52k [00:00<00:00, 26.5MB/s]
model.safetensors: 100%|███████████████████████████████████████████████████████████████████████████████████| 1.71G/1.71G [01:33<00:00, 18.3MB/s]
02:35:05-332247 INFO     LDM: LatentDiffusion: mode=eps
02:35:05-333195 INFO     LDM: DiffusionWrapper params=859.52M
02:35:05-334071 INFO     LDM: [2;36m02:33:55-715782[0m[2;36m [0m[34mINFO    [0m MOTD: N/A
02:35:05-334801 INFO     LDM: [2;36m02:33:57-118661[0m[2;36m [0m[34mINFO    [0m Browser session: [33muser[0m=[3;35mNone[0m
                         [33mclient[0m=[1;92m127[0m[1;92m.0.0.1[0m [33magent[0m=[35mMozilla[0m/[1;36m5.0[0m [1m([0mWindows NT [1;36m10.0[0m;
                         Win64; x64; rv:[1;36m131.0[0m[1m)[0m Gecko/[1;36m20100101[0m
02:35:05-336218 INFO     LDM: [2;36m                [0m         Firefox/[1;36m131.0[0m
02:35:05-337278 INFO     Autodetect model: detect="Stable Diffusion" class=StableDiffusionPipeline
                         file="/home/arc22/sdx/ca/models/Stable-diffusion/CheckpointYesmix_v50.safetensors" size=2034MB
02:35:07-726712 INFO     Cross-attention: optimization=Scaled-Dot-Product
02:35:08-045191 INFO     Load network: type=embeddings loaded=0 skipped=0 time=0.31
02:35:08-049251 INFO     Model loaded in 105.97 create=103.22 apply=0.51 vae=0.96 move=0.91 embeddings=0.32
02:35:08-240934 INFO     Model load finished: {'ram': {'used': 6.29, 'total': 23.32}, 'gpu': {'used': 2.03, 'total': 7.91}, 'retries': 0, 'oom':
                         0} cached=0
02:35:08-271600 INFO     Startup time: 113.08 torch=3.18 gradio=0.64 libraries=1.11 samplers=0.09 extensions=0.69 ui-networks=0.08
                         ui-txt2img=0.05 ui-settings=0.15 ui-extensions=0.08 launch=0.30 api=0.10 app-started=0.15 checkpoint=106.19

Backend

Original

UI

Standard

Branch

Master

Model

StableDiffusion 1.5

Acknowledgements

• [X] I have read the above and searched for existing issues
• [X] I confirm that this is classified correctly and its not an extension issue

[vladmandic]

pls run without your negative prompt - its really bad and its causing massive burn. if issue persists, then we can look at it further.

[cyatarow]

I tried it again with no negative prompts, but the issue persists.

[cyatarow]

(Additional Information) I tried an older version of SD.Next (updated=2024-07-22 hash=25c3c610 branch=master), and the issue did not happen.

Model-> YesMix v5.0 Prompt-> 1girl, forest, masterpiece, best quality Negative-> (none) Size-> 512x768 Guidance scale-> 8 Steps-> 25 Sampling method-> DPM++ 2M Seed-> 2368381021

[vladmandic]

@Disty0 can this be ipex version or dtype related? i cant reproduce on cuda.

[Disty0]

Can you try this on the dev branch? Might be because of the last attention mask fix didn't make it into master branch yet.

[Disty0]

Tried it with the master and dev, I can't reproduce this on either.

Giving it another try, it is an issue with that model and the specific seed.

2368381020	2368381021	2368381022

Here is the broken seed with another model:

[Disty0]

Using PyTorch 2.5+ also fixes this. This issue is probably related to this: https://github.com/intel/intel-extension-for-pytorch/issues/529

You can manually install PyTorch 2.5 by deleting the venv folder first, then exporting this command to tell SDNext to use different PyTorch but you will lose around 40% of the performance with PyTorch 2.5 compared to IPEX 2.3:

export TORCH_COMMAND="torch torchvision --index-url https://download.pytorch.org/whl/test/xpu"
./webui.sh --use-ipex

PyTorch 2.5 with the broken seed:

[Disty0]

Closing this off since it is an edge case with a specific model and specific seed and also an upstream IPEX issue.

[cyatarow]

Using PyTorch 2.5+ also fixes this.

PyTorch 2.5.1 fixed it. Thank you!