search

vladmandic / automatic

SD.Next: Advanced Implementation Generative Image Models
https://github.com/vladmandic/automatic
GNU Affero General Public License v3.0
5.76k stars 431 forks source link

[Feature]: xyz grid support multi-dimensional search&replace #3502

Closed SAC020 closed 2 days ago

SAC020 commented 1 month ago

Issue Description

xyz search and replace with only x activated, works fine, i.e. it applies lora <000002000>, then <000004000>

Prompt is:

photo of man on the street

comic book style <lora:Comic_book_V2:1.5> ,

<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>
17:29:23-790923 INFO     XYZ grid start: images=8 grid=1 shape=8x1 cells=1 steps=320
17:29:23-792918 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000002000:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:29:24-741893 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000002000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:29:25-582644 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:29:25-604586 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.81
17:29:25-609572 INFO     Base: class=FluxPipeline
17:29:25-611567 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:29:25-613561 DEBUG    Torch generator: device=cuda seeds=[2218383567]
17:29:25-615556 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.28it/s ████████████████████████████▉      88% 35/40 00:30 00:03 Base17:29:59-974877 DEBUG    Server: alive=True jobs=1 requests=2486 uptime=4176 memory=32.11/63.92
                         backend=Backend.DIFFUSERS state=idle
Progress  1.15it/s █████████████████████████████████ 100% 40/40 00:34 00:00 Base
17:30:05-284987 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.939
17:30:05-528334 INFO     Processed: images=1 its=0.96 time=41.73 timers={'init': 1.81, 'callback': 6.77, 'pipeline':
                         37.67, 'decode': 2.21, 'post': 0.03} memory={'ram': {'used': 37.84, 'total': 63.92}, 'gpu':
                         {'used': 9.0, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:30:05-530330 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000004000:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:30:05-926271 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000004000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:30:07-296869 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000004000" type={'ModuleTypeLora'} keys=494
17:30:07-316816 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000004000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.79
17:30:07-319808 INFO     Base: class=FluxPipeline
17:30:07-321803 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:30:07-323797 DEBUG    Torch generator: device=cuda seeds=[2218383567]
17:30:07-324795 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.00s/it █████████████████████████████████ 100% 40/40 00:40 00:00 Base
17:30:51-763889 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.897
17:30:51-973328 INFO     Processed: images=1 its=0.86 time=46.44 timers={'init': 1.79, 'callback': 6.73, 'pipeline':
                         42.47, 'decode': 2.15, 'post': 0.02} memory={'ram': {'used': 37.91, 'total': 63.92}, 'gpu':
                         {'used': 8.99, 'total': 23.99}, 'retries': 0, 'oom': 0}

Same prompt, same x, but now with y activated as well, x gets ignored; it "says" it will replace the string, but applies the same lora <000002000> over and over again.

17:43:39-329259 INFO     XYZ grid start: images=24 grid=1 shape=8x3 cells=1 steps=960
17:43:39-331254 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000002000:1.5>"
17:43:39-332251 DEBUG    XYZ grid apply prompt: "comic book style <lora:Comic_book_V2:1.5>"="comic book style
                         <lora:Comic_book_V2:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:43:39-848385 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000002000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:43:40-737007 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:43:40-757952 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.42
17:43:40-763936 INFO     Base: class=FluxPipeline
17:43:40-766928 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:43:40-769920 DEBUG    Torch generator: device=cuda seeds=[3557369830]
17:43:40-770917 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.04s/it ██████▊                             20% 8/40 00:15 00:33 Base17:43:59-845513 DEBUG    Server: alive=True jobs=1 requests=2916 uptime=5016 memory=46.89/63.92
                         backend=Backend.DIFFUSERS state=idle
Progress  1.04s/it █████████████████████████████████ 100% 40/40 00:41 00:00 Base
17:44:30-375357 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=5.088
17:44:30-501021 INFO     Processed: images=1 its=0.78 time=51.17 timers={'init': 1.43, 'callback': 7.24, 'pipeline':
                         44.13, 'decode': 5.56, 'post': 0.03} memory={'ram': {'used': 41.08, 'total': 63.92}, 'gpu':
                         {'used': 5.13, 'total': 23.99}, 'retries': 0, 'oom': 0}
**17:44:30-505010 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-dalle-flux-v3-colab-capt
                         ions_000004000:1.5>"**
17:44:30-506009 DEBUG    XYZ grid apply prompt: "comic book style <lora:Comic_book_V2:1.5>"="comic book style
                         <lora:Comic_book_V2:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:44:30-996210 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_**000002000**.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:44:32-052895 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:44:32-072841 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.56
17:44:32-077828 INFO     Base: class=FluxPipeline
17:44:32-079822 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:44:32-081846 DEBUG    Torch generator: device=cuda seeds=[3557369830]
17:44:32-082815 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}
Progress  1.13it/s █████████████████████████████████ 100% 40/40 00:35 00:00 Base
17:45:12-477798 DEBUG    VAE decode: vae name="default" dtype=torch.bfloat16 device=cuda:0 upcast=False slicing=True
                         tiling=True latents shape=torch.Size([1, 16, 128, 128]) dtype=torch.bfloat16 device=cuda:0
                         time=1.888
17:45:12-706216 INFO     Processed: images=1 its=0.95 time=42.20 timers={'init': 1.57, 'callback': 7.12, 'pipeline':
                         38.37, 'decode': 2.22, 'post': 0.03} memory={'ram': {'used': 37.78, 'total': 63.92}, 'gpu':
                         {'used': 9.11, 'total': 23.99}, 'retries': 0, 'oom': 0}
17:45:12-713196 DEBUG    XYZ grid apply prompt:
                         "<lora:sacbf-dalle-flux-v3-colab-captions_000002000:1.5>"="<lora:sacbf-flux-dalle-v1_epoch_25:0
                         .7>"
17:45:12-715163 DEBUG    XYZ grid apply prompt: "comic book style <lora:Comic_book_V2:1.5>"="comic book style
                         <lora:Comic_book_V2:1.5>"
Load network: C:\ai\automatic\models\Lora\Comic_book_V2.safetensors ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/211.0 MB -:--:--
17:45:13-119224 DEBUG    LoRA name="Comic_book_V2" type={'ModuleTypeLora'} keys=338
Load network: C:\ai\automatic\models\Lora\sacbf-dalle-flux-v3-colab-captions_000002000.safetensors ━━━━━━ 0.0/1… -:--:--
                                                                                                          MB
17:45:13-874205 DEBUG    LoRA name="sacbf-dalle-flux-v3-colab-captions_000002000" type={'ModuleTypeLora'} keys=494
17:45:13-897144 INFO     Load network: type=LoRA apply=['Comic_book_V2', 'sacbf-dalle-flux-v3-colab-captions_000002000']
                         te=[1.5, 1.5] unet=[[1.5, 1.5, 1.5], [1.5, 1.5, 1.5]] dims=[None, None] load=1.18
17:45:13-901134 INFO     Base: class=FluxPipeline
17:45:13-902130 DEBUG    Sampler: sampler=default class=FlowMatchEulerDiscreteScheduler: {'num_train_timesteps': 1000,
                         'shift': 3.0, 'use_dynamic_shifting': True, 'base_shift': 0.5, 'max_shift': 1.15,
                         'base_image_seq_len': 256, 'max_image_seq_len': 4096}
17:45:13-905123 DEBUG    Torch generator: device=cuda seeds=[3557369830]
17:45:13-907117 DEBUG    Diffuser pipeline: FluxPipeline task=DiffusersTaskType.TEXT_2_IMAGE batch=1/1x1 set={'prompt':
                         1, 'guidance_scale': 4, 'num_inference_steps': 40, 'output_type': 'latent', 'width': 1024,
                         'height': 1024, 'parser': 'Fixed attention'}

It happens both with "script xyz" and "global / extension xyz".

Version Platform Description

PS C:\ai\automatic> .\webui --debug --medvram Using VENV: C:\ai\automatic\venv 17:53:40-393526 INFO Starting SD.Next 17:53:40-396520 INFO Logger: file="C:\ai\automatic\sdnext.log" level=DEBUG size=65 mode=create 17:53:40-397516 INFO Python: version=3.11.9 platform=Windows bin="C:\ai\automatic\venv\Scripts\Python.exe" venv="C:\ai\automatic\venv" 17:53:40-608585 INFO Version: app=sd.next updated=2024-10-21 hash=cc4f4819 branch=dev url=https://github.com/vladmandic/automatic/tree/dev ui=dev 17:53:41-322526 INFO Repository latest available 5c684cb07606c423feaa06abe83608b287e9b68a 2024-10-17T04:21:39Z 17:53:41-331950 INFO Platform: arch=AMD64 cpu=Intel64 Family 6 Model 165 Stepping 5, GenuineIntel system=Windows release=Windows-10-10.0.22631-SP0 python=3.11.9

Relevant log output

No response

Backend

Diffusers

UI

Standard

Branch

Dev

Model

Other

Acknowledgements

SAC020 commented 1 month ago

Not sure why this is marked as an enhancement, xyz used to support multi-dimensional S/R since forever (I have been using it since forever)

SAC020 commented 3 weeks ago

Hi! Checking if there's anything planned on this, it's would really help my process of testing lora combos. Thanks!

uwidev commented 1 week ago

If it wasn't confirmed yet, this also does not work on the Z-axis. I was thinking maybe a workaround could be X-Z, but that doesn't work either. I was also thinking maybe this was something specific to LoRAs in the event it treated LoRAs differently. Nope, for standard tags it also does not search and replace correctly.

vladmandic commented 2 days ago

done. image

SAC020 commented 2 days ago

Thank you!