search

JasonS09 / comfy_sd_krita_plugin

Make AI art between canvas and nodes with Krita.
MIT License
148 stars 7 forks source link

LoRAs in subfolders not applied #20

Closed IanLiotta closed 9 months ago

IanLiotta commented 1 year ago

Describe the bug

Attempting to use a LoRA in a subfolder, such as <lora:Style\add_detail:1> produces no error, but the LoRA is not applied.

To Reproduce Steps to reproduce the behavior:

  1. Create a txt2img prompt with a LoRA that has a significant effect such as <lora:Style\add_detail:1>
  2. Generate the image
  3. Create a txt2img prompt with the same seed and a LoRA such as
  4. Generate the image and compare the output

Desktop (please complete the following information):

ElliottLester commented 1 year ago

see pull request #18 as a work around you can pass the full path to the lora inside the macro eg 

<lora:Style\add_detail:1>

Check:

I have updated to #18 to ignore the case

IanLiotta commented 1 year ago

Ah, OK, a few things: I had been writing the statement like <lora:Misc\add_detail:1> and it was not working but not giving an error. Trying to type <lora:Misc/add_detail:1> produces the following error:

Failed to validate prompt for output SaveImage:
* LoraLoader LoraLoader+1:
  - Value not in list: lora_name: 'Misc/add_detail.safetensors' not in (list of length 106)
Output will be ignored
invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}

However! Typing lora:Misc/add_detail:1 without the <> works correctly, applying the LoRA from the subfolder.

ElliottLester commented 1 year ago

The error you are getting means it's trying to load the lora from that path but it's not there. Open the Comfy web UI and add node -> loaders -> "load lora" and check the list in the lora_name to see if Misc/add_detail is in that list

As for you're success case: The regex that matches the lora is "<lora:([=\[\] /\w\d.-]+):([\d.]+)>" which requires the <> I think the lora:Misc/add_detail:1 is just being pass as a string to the prompt, so no error

IanLiotta commented 1 year ago

It appears as Misc\add_detail with the backslash in the WebUI LoraLoader.

ElliottLester commented 1 year ago

ah ok so you are running the server on windows and krita on windows so all the slashes are backwards

ElliottLester commented 1 year ago

I just pushed a change to support both slashes into my auto complete branch. if it's not too much trouble could you try testing the code in my PR? git clone -b lora_autocomplete https://github.com/ElliottLester/comfy_sd_krita_plugin.git

it should support both <lora:add_detail:1> <lora:Misc\add_detail:1>

IanLiotta commented 1 year ago

Your PR is working correctly. I'm getting the same results from both the WebUI with a LoRA Loader and Krita plugin with a prompt containing <lora:Misc\add_detail:1>

ElliottLester commented 1 year ago

@JasonS09 #18 was merged so I think this can be resolved

IanLiotta commented 1 year ago

I've run into another problem with the backlash notation. Trying to use a LoRA that starts with the letter 'g' causes an error: 

error
Python 3.8.1: C:\Program Files\Krita (x64)\bin\krita.exe
Sun Sep  3 14:39:59 2023

A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\pages\txt2img.py in <lambda>()
   43         toggle_highres(not script.cfg("disable_sddebz_highres", bool))
   44 
   45         self.btn.released.connect(lambda: script.action_txt2img())
   46         self.get_workflow_btn.released.connect(
   47             lambda: get_workflow(script.cfg, script.action_get_workflow, "txt2img")
self undefined
global script = <krita_comfy.script.Script object>
script.action_txt2img = <bound method Script.action_txt2img of <krita_comfy.script.Script object>>

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\script.py in action_txt2img(self=<krita_comfy.script.Script object>)
  567             return
  568         self.adjust_selection()
  569         self.apply_txt2img()
  570 
  571     def action_img2img(self):
self = <krita_comfy.script.Script object>
self.apply_txt2img = <bound method Script.apply_txt2img of <krita_comfy.script.Script object>>

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\script.py in apply_txt2img(self=<krita_comfy.script.Script object>)
  342 
  343         sel_image = self.get_selection_image()
  344         self.client.post_txt2img(
  345             cb, self.width, self.height, sel_image,
  346             self.get_controlnet_input_images(sel_image)
self = <krita_comfy.script.Script object>
self.client = <krita_comfy.client.Client object>
self.client.post_txt2img = <bound method Client.post_txt2img of <krita_comfy.client.Client object>>
cb = <function Script.basic_callback_crafter.<locals>.cb>
self.width = 512
self.height = 512
sel_image = <PyQt5.QtGui.QImage object>
self.get_controlnet_input_images = <bound method Script.get_controlnet_input_images of <krita_comfy.script.Script object>>

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py in post_txt2img(self=<krita_comfy.client.Client object>, cb=<function Script.basic_callback_crafter.<locals>.cb>, width=512, height=512, src_img=<PyQt5.QtGui.QImage object>, controlnet_src_imgs={})
 1233                     self.upscale_latent(params, width, height, seed, "txt2img")
 1234 
 1235             self.loadLoRAs(params, "txt2img")
 1236             self.apply_controlnet(params, controlnet_src_imgs)
 1237         else:
self = <krita_comfy.client.Client object>
self.loadLoRAs = <bound method Client.loadLoRAs of <krita_comfy.client.Client object>>
params = {'3': {'class_type': 'KSampler', 'inputs': {'cfg': 8, 'denoise': 1, 'latent_image': ['5', 0], 'model': ['4', 0], 'negative': ['7', 0], 'positive': ['6', 0], 'sampler_name': 'dpmpp_2m_sde', 'scheduler': 'exponential', 'seed': 4773042294970426912, 'steps': 40}}, '4': {'class_type': 'CheckpointLoaderSimple', 'inputs': {'ckpt_name': 'aZovyaRPGArtistTools_v3.safetensors'}}, '5': {'class_type': 'EmptyLatentImage', 'inputs': {'batch_size': 1, 'height': 512, 'width': 512}}, '6': {'class_type': 'CLIPTextEncode', 'inputs': {'clip': ['ClipSetLastLayer', 0], 'text': 'masterpiece anime toon best quality\n\nblue goblin\nforest night stars moon'}}, '7': {'class_type': 'CLIPTextEncode', 'inputs': {'clip': ['ClipSetLastLayer', 0], 'text': 'lowres, bad anatomy, bad hands, text, error, mis... embedding:bad-hands-5, embedding:boring_e621_v4,'}}, 'ClipSetLastLayer': {'class_type': 'CLIPSetLastLayer', 'inputs': {'clip': ['4', 1], 'stop_at_clip_layer': -2}}, 'SaveImage': {'class_type': 'SaveImage', 'inputs': {'filename_prefix': 'ComfyUI', 'images': ['VAEDecode', 0]}}, 'VAEDecode': {'class_type': 'VAEDecode', 'inputs': {'samples': ['3', 0], 'vae': ['VAELoader', 0]}}, 'VAELoader': {'class_type': 'VAELoader', 'inputs': {'vae_name': 'vaeFtMse840000Ema_v10.safetensors'}}}

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py in loadLoRAs(self=<krita_comfy.client.Client object>, params={'3': {'class_type': 'KSampler', 'inputs': {'cfg': 8, 'denoise': 1, 'latent_image': ['5', 0], 'model': ['4', 0], 'negative': ['7', 0], 'positive': ['6', 0], 'sampler_name': 'dpmpp_2m_sde', 'scheduler': 'exponential', 'seed': 4773042294970426912, 'steps': 40}}, '4': {'class_type': 'CheckpointLoaderSimple', 'inputs': {'ckpt_name': 'aZovyaRPGArtistTools_v3.safetensors'}}, '5': {'class_type': 'EmptyLatentImage', 'inputs': {'batch_size': 1, 'height': 512, 'width': 512}}, '6': {'class_type': 'CLIPTextEncode', 'inputs': {'clip': ['ClipSetLastLayer', 0], 'text': 'masterpiece anime toon best quality\n\nblue goblin\nforest night stars moon'}}, '7': {'class_type': 'CLIPTextEncode', 'inputs': {'clip': ['ClipSetLastLayer', 0], 'text': 'lowres, bad anatomy, bad hands, text, error, mis... embedding:bad-hands-5, embedding:boring_e621_v4,'}}, 'ClipSetLastLayer': {'class_type': 'CLIPSetLastLayer', 'inputs': {'clip': ['4', 1], 'stop_at_clip_layer': -2}}, 'SaveImage': {'class_type': 'SaveImage', 'inputs': {'filename_prefix': 'ComfyUI', 'images': ['VAEDecode', 0]}}, 'VAEDecode': {'class_type': 'VAEDecode', 'inputs': {'samples': ['3', 0], 'vae': ['VAELoader', 0]}}, 'VAELoader': {'class_type': 'VAELoader', 'inputs': {'vae_name': 'vaeFtMse840000Ema_v10.safetensors'}}}, mode='txt2img', connect_last_lora_outputs=True)
  489             for match in matches:
  490                 # Extract the lora name and the strength number from the match
  491                 lora_name = self.auto_complete_LoRA(match[0])
  492                 strength_number = float(match[1])
  493 
lora_name undefined
self = <krita_comfy.client.Client object>
self.auto_complete_LoRA = <bound method Client.auto_complete_LoRA of <krita_comfy.client.Client object>>
match = (r'Styles\gobbo', '0.5')

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py in auto_complete_LoRA(self=<krita_comfy.client.Client object>, name=r'Styles\gobbo')
  453     def auto_complete_LoRA(self, name: str):
  454         lora_list = [ re.sub(".safetensors$", "", l, flags=re.I) for l in self.cfg("sd_lora_list", str)]
  455         viable_loras = [l for l in lora_list if re.search(name+"$", l, flags=re.I)]
  456         if (len(viable_loras) == 1 and name == viable_loras[0]):
  457             return name
viable_loras undefined
l undefined
lora_list = [r'Misc\000_aiMeishi_yfdesign_v02-07', r'Misc\EdobHorse_v1.0', r'Misc\add_detail', r'Misc\bloodvessel', r'Misc\color_slider_v10', r'Misc\cutememe02', r'Misc\cyber_room-10', r'Misc\fluffyrock-quality-tags-v3.0-vpred', r'Misc\hamichichiB', r'Misc\more_details', r'Misc\n7362z', r'Styles\Nendroid-000030', r'Styles\Rizdraws [MockAi - v1.0]', r'Styles\angelauxes-08', r'Styles\concept_exploding_clothes_v1_768px', r'Styles\gasmask', r'Styles\gobbo', r'Styles\latex_huger_c7-1+76-1+64-4', r'Styles\lineart + flat colors v2', r'Styles\popupparade-10', ...]
global re = <module 're' from 'C:\\Program Files\\Krita (x64)\\python\\python38.zip\\re.pyc'>
re.search = <function search>
name = r'Styles\gobbo'
flags undefined
re.I = re.IGNORECASE

 C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py in <listcomp>(.0=<list_iterator object>)
  453     def auto_complete_LoRA(self, name: str):
  454         lora_list = [ re.sub(".safetensors$", "", l, flags=re.I) for l in self.cfg("sd_lora_list", str)]
  455         viable_loras = [l for l in lora_list if re.search(name+"$", l, flags=re.I)]
  456         if (len(viable_loras) == 1 and name == viable_loras[0]):
  457             return name
viable_loras undefined
l = r'Misc\000_aiMeishi_yfdesign_v02-07'
lora_list undefined
global re = <module 're' from 'C:\\Program Files\\Krita (x64)\\python\\python38.zip\\re.pyc'>
re.search = <function search>
name = r'Styles\gobbo'
flags undefined
re.I = re.IGNORECASE

 C:\Program Files\Krita (x64)\re.py in search(pattern=r'Styles\gobbo$', string=r'Misc\000_aiMeishi_yfdesign_v02-07', flags=re.IGNORECASE)

 C:\Program Files\Krita (x64)\re.py in _compile(pattern=r'Styles\gobbo$', flags=2)

 C:\Program Files\Krita (x64)\sre_compile.py in compile(p=r'Styles\gobbo$', flags=2)

 C:\Program Files\Krita (x64)\sre_parse.py in parse(str=r'Styles\gobbo$', flags=2, state=<sre_parse.State object>)

 C:\Program Files\Krita (x64)\sre_parse.py in _parse_sub(source=<sre_parse.Tokenizer object>, state=<sre_parse.State object>, verbose=0, nested=0)

 C:\Program Files\Krita (x64)\sre_parse.py in _parse(source=<sre_parse.Tokenizer object>, state=<sre_parse.State object>, verbose=0, nested=1, first=True)

 C:\Program Files\Krita (x64)\sre_parse.py in _escape(source=<sre_parse.Tokenizer object>, escape=r'\g', state=<sre_parse.State object>)

error: bad escape \g at position 6
    __cause__ = None
    __class__ = <class 're.error'>
    __context__ = None
    __delattr__ = <method-wrapper '__delattr__' of error object>
    __dict__ = {'colno': 7, 'lineno': 1, 'msg': r'bad escape \g', 'pattern': r'Styles\gobbo$', 'pos': 6}
    __dir__ = <built-in method __dir__ of error object>
    __doc__ = None
    __eq__ = <method-wrapper '__eq__' of error object>
    __format__ = <built-in method __format__ of error object>
    __ge__ = <method-wrapper '__ge__' of error object>
    __getattribute__ = <method-wrapper '__getattribute__' of error object>
    __gt__ = <method-wrapper '__gt__' of error object>
    __hash__ = <method-wrapper '__hash__' of error object>
    __init__ = <bound method error.__init__ of error('bad escape \\g at position 6')>
    __init_subclass__ = <built-in method __init_subclass__ of type object>
    __le__ = <method-wrapper '__le__' of error object>
    __lt__ = <method-wrapper '__lt__' of error object>
    __module__ = 're'
    __ne__ = <method-wrapper '__ne__' of error object>
    __new__ = <built-in method __new__ of type object>
    __reduce__ = <built-in method __reduce__ of error object>
    __reduce_ex__ = <built-in method __reduce_ex__ of error object>
    __repr__ = <method-wrapper '__repr__' of error object>
    __setattr__ = <method-wrapper '__setattr__' of error object>
    __setstate__ = <built-in method __setstate__ of error object>
    __sizeof__ = <built-in method __sizeof__ of error object>
    __str__ = <method-wrapper '__str__' of error object>
    __subclasshook__ = <built-in method __subclasshook__ of type object>
    __suppress_context__ = False
    __traceback__ = <traceback object>
    __weakref__ = None
    args = (r'bad escape \g at position 6',)
    colno = 7
    lineno = 1
    msg = r'bad escape \g'
    pattern = r'Styles\gobbo$'
    pos = 6
    with_traceback = <built-in method with_traceback of error object>

The above is a description of an error in a Python program.  Here is
the original traceback:

Traceback (most recent call last):
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\pages\txt2img.py", line 45, in <lambda>
    self.btn.released.connect(lambda: script.action_txt2img())
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\script.py", line 569, in action_txt2img
    self.apply_txt2img()
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\script.py", line 344, in apply_txt2img
    self.client.post_txt2img(
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py", line 1235, in post_txt2img
    self.loadLoRAs(params, "txt2img")
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py", line 491, in loadLoRAs
    lora_name = self.auto_complete_LoRA(match[0])
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py", line 455, in auto_complete_LoRA
    viable_loras = [l for l in lora_list if re.search(name+"$", l, flags=re.I)]
  File "C:\Users\mix\AppData\Roaming\krita\pykrita\krita_comfy\client.py", line 455, in <listcomp>
    viable_loras = [l for l in lora_list if re.search(name+"$", l, flags=re.I)]
  File "re.py", line 199, in search
  File "re.py", line 302, in _compile
  File "sre_compile.py", line 764, in compile
  File "sre_parse.py", line 948, in parse
  File "sre_parse.py", line 443, in _parse_sub
  File "sre_parse.py", line 525, in _parse
  File "sre_parse.py", line 426, in _escape
re.error: bad escape \g at position 6
ElliottLester commented 1 year ago

doh, This should fix it git clone -b lora_regex_fix https://github.com/ElliottLester/comfy_sd_krita_plugin.git opened as #21

JasonS09 commented 1 year ago

@IanLiotta how about now? Is it working correctly?

IanLiotta commented 1 year ago

Elliott's fix is working for me, yes.

jiushier commented 1 year ago

Can Krita Android be used？

ElliottLester commented 1 year ago

Can Krita Android be used？

https://krita-artists.org/t/any-reason-for-no-plugin-support-on-android/55749/5

No pyqt5 on android seems to be the reason.