search

vladmandic / automatic

SD.Next: Advanced Implementation of Stable Diffusion and other Diffusion-based generative image models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.67k stars 418 forks source link

[Feature]: xyz grid support multi-dimensional search&replace #3502

Open SAC020 opened 1 week ago

SAC020 commented 1 week ago

Issue Description

xyz search and replace with only x activated, works fine, i.e. it applies lora <000002000>, then <000004000>

Prompt is:

photo of man on the street

comic book style <lora:Comic_book_V2:1.5> ,

<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>
17:29:23-790923 INFO     XYZ grid start: images=8 grid=1 shape=8x1 cells=1 steps=320
17:29:23-792918 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000002000:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:29:24-741893 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000002000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:29:25-582644 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:29:25-604586 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.81
17:29:25-609572 INFO     Base: class=FluxPipeline
17:29:25-611567 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:29:25-613561 DEBUG    Torch generator: device=cuda seeds=[2218383567]
17:29:25-615556 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.28it/s ████████████████████████████▉      88% 35/40 00:30 00:03 Base17:29:59-974877 DEBUG    Server: alive=True jobs=1 requests=2486 uptime=4176 memory=32.11/63.92
                         backend=Backend.DIFFUSERS state=idle
Progress  1.15it/s █████████████████████████████████ 100% 40/40 00:34 00:00 Base
17:30:05-284987 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.939
17:30:05-528334 INFO     Processed: images=1 its=0.96 time=41.73 timers={'init': 1.81, 'callback': 6.77, 'pipeline':
                         37.67, 'decode': 2.21, 'post': 0.03} memory={'ram': {'used': 37.84, 'total': 63.92}, 'gpu':
                         {'used': 9.0, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:30:05-530330 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000004000:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:30:05-926271 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000004000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:30:07-296869 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000004000" type={'ModuleTypeLora'} keys=494
17:30:07-316816 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000004000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.79
17:30:07-319808 INFO     Base: class=FluxPipeline
17:30:07-321803 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:30:07-323797 DEBUG    Torch generator: device=cuda seeds=[2218383567]
17:30:07-324795 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.00s/it █████████████████████████████████ 100% 40/40 00:40 00:00 Base
17:30:51-763889 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.897
17:30:51-973328 INFO     Processed: images=1 its=0.86 time=46.44 timers={'init': 1.79, 'callback': 6.73, 'pipeline':
                         42.47, 'decode': 2.15, 'post': 0.02} memory={'ram': {'used': 37.91, 'total': 63.92}, 'gpu':
                         {'used': 8.99, 'total': 23.99}, 'retries': 0, 'oom': 0}

Same prompt, same x, but now with y activated as well, x gets ignored; it "says" it will replace the string, but applies the same lora <000002000> over and over again.

17:43:39-329259 INFO     XYZ grid start: images=24 grid=1 shape=8x3 cells=1 steps=960
17:43:39-331254 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000002000:1.5>"
17:43:39-332251 DEBUG    XYZ grid apply prompt: "comic book style <lora:Comic_book_V2:1.5>"="comic book style
                         <lora:Comic_book_V2:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:43:39-848385 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000002000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:43:40-737007 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:43:40-757952 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.42
17:43:40-763936 INFO     Base: class=FluxPipeline
17:43:40-766928 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:43:40-769920 DEBUG    Torch generator: device=cuda seeds=[3557369830]
17:43:40-770917 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.04s/it ██████▊                             20% 8/40 00:15 00:33 Base17:43:59-845513 DEBUG    Server: alive=True jobs=1 requests=2916 uptime=5016 memory=46.89/63.92
                         backend=Backend.DIFFUSERS state=idle
Progress  1.04s/it █████████████████████████████████ 100% 40/40 00:41 00:00 Base
17:44:30-375357 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=5.088
17:44:30-501021 INFO     Processed: images=1 its=0.78 time=51.17 timers={'init': 1.43, 'callback': 7.24, 'pipeline':
                         44.13, 'decode': 5.56, 'post': 0.03} memory={'ram': {'used': 41.08, 'total': 63.92}, 'gpu':
                         {'used': 5.13, 'total': 23.99}, 'retries': 0, 'oom': 0}
**17:44:30-505010 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000004000:1.5>"**
17:44:30-506009 DEBUG    XYZ grid apply prompt: "comic book style <lora:Comic_book_V2:1.5>"="comic book style
                         <lora:Comic_book_V2:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:44:30-996210 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_**000002000**.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:44:32-052895 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:44:32-072841 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.56
17:44:32-077828 INFO     Base: class=FluxPipeline
17:44:32-079822 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:44:32-081846 DEBUG    Torch generator: device=cuda seeds=[3557369830]
17:44:32-082815 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.13it/s █████████████████████████████████ 100% 40/40 00:35 00:00 Base
17:45:12-477798 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.888
17:45:12-706216 INFO     Processed: images=1 its=0.95 time=42.20 timers={'init': 1.57, 'callback': 7.12, 'pipeline':
                         38.37, 'decode': 2.22, 'post': 0.03} memory={'ram': {'used': 37.78, 'total': 63.92}, 'gpu':
                         {'used': 9.11, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:45:12-713196 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-flux-dalle-v1_epoch_25:0
                         .7>"
17:45:12-715163 DEBUG    XYZ grid apply prompt: "comic book style <lora:Comic_book_V2:1.5>"="comic book style
                         <lora:Comic_book_V2:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:45:13-119224 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000002000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:45:13-874205 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:45:13-897144 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.18
17:45:13-901134 INFO     Base: class=FluxPipeline
17:45:13-902130 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:45:13-905123 DEBUG    Torch generator: device=cuda seeds=[3557369830]
17:45:13-907117 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}

It happens both with "script xyz" and "global / extension xyz".

Version Platform Description

PS C:\ai\automatic> .\webui --debug --medvram Using VENV: C:\ai\automatic\venv 17:53:40-393526 INFO Starting SD.Next 17:53:40-396520 INFO Logger: file="C:\ai\automatic\sdnext.log" level=DEBUG size=65 mode=create 17:53:40-397516 INFO Python: version=3.11.9 platform=Windows bin="C:\ai\automatic\venv\Scripts\Python.exe" venv="C:\ai\automatic\venv" 17:53:40-608585 INFO Version: app=sd.next updated=2024-10-21 hash=cc4f4819 branch=dev url=https://github.com/vladmandic/automatic/tree/dev ui=dev 17:53:41-322526 INFO Repository latest available 5c684cb07606c423feaa06abe83608b287e9b68a 2024-10-17T04:21:39Z 17:53:41-331950 INFO Platform: arch=AMD64 cpu=Intel64 Family 6 Model 165 Stepping 5, GenuineIntel system=Windows release=Windows-10-10.0.22631-SP0 python=3.11.9

Relevant log output

No response

Backend

Diffusers

UI

Standard

Branch

Dev

Model

Other

Acknowledgements

SAC020 commented 6 days ago

Not sure why this is marked as an enhancement, xyz used to support multi-dimensional S/R since forever (I have been using it since forever)