ljleb / prompt-fusion-extension

auto1111 webui extension for all sorts of prompt interpolations!
MIT License
259 stars 16 forks source link

ADetailer may be sometimes incompatible; 1.6.0 #63

Closed pmcculler closed 11 months ago

pmcculler commented 11 months ago

I think having ADetailer enabled causes this gen-halting error frequently.

Traceback (most recent call last):
  File "D:\Users\pmccu\stable-diffusion-webui\modules\call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "D:\Users\pmccu\stable-diffusion-webui\modules\call_queue.py", line 36, in f
    res = func(*args, **kwargs)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
    processed = processing.process_images(p)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 732, in process_images
    res = process_images_inner(p)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 856, in process_images_inner
    p.setup_conds()
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 1309, in setup_conds
    super().setup_conds()
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 470, in setup_conds
    self.c = self.get_conds_with_caching(prompt_parser.get_multicond_learned_conditioning, prompts, total_steps, [self.cached_c], self.extra_network_data)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 455, in get_conds_with_caching
    cache[1] = function(shared.sd_model, required_prompts, steps, hires_steps, shared.opts.use_old_scheduling)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\hijacker.py", line 12, in wrapper
    return function(*args, **kwargs, original_function=self.__original_functions[attribute])
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 92, in _hijacked_get_multicond_learned_conditioning
    res = original_function(*args, **kwargs)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\prompt_parser.py", line 262, in get_multicond_learned_conditioning
    learned_conditioning = get_learned_conditioning(model, prompt_flat_list, steps, hires_steps, use_old_scheduling)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\hijacker.py", line 12, in wrapper
    return function(*args, **kwargs, original_function=self.__original_functions[attribute])
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 70, in _hijacked_get_learned_conditioning
    schedules = [_sample_tensor_schedules(cond_tensor, real_total_steps, is_hires=hires_steps is not None)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 70, in <listcomp>
    schedules = [_sample_tensor_schedules(cond_tensor, real_total_steps, is_hires=hires_steps is not None)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 127, in _sample_tensor_schedules
    schedule_cond = tensor.interpolate(params, origin_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 21, in interpolate
    cond = self.interpolate_rec(params, 0, origin_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 41, in interpolate_rec
    control_points = [
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 42, in <listcomp>
    InterpolationTensor(sub_tensor, self.__interpolation_functions, self.__empty_cond).interpolate_rec(params, axis + 1, origin_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 41, in interpolate_rec
    control_points = [
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 42, in <listcomp>
    InterpolationTensor(sub_tensor, self.__interpolation_functions, self.__empty_cond).interpolate_rec(params, axis + 1, origin_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 41, in interpolate_rec
    control_points = [
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 42, in <listcomp>
    InterpolationTensor(sub_tensor, self.__interpolation_functions, self.__empty_cond).interpolate_rec(params, axis + 1, origin_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 47, in interpolate_rec
    control_points[i] = InterpolationTensor(control_points[i], nested_functions, self.__empty_cond).interpolate_rec(params, 0, origin_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 32, in interpolate_rec
    if schedule.end_at_step >= params.step:
AttributeError: 'list' object has no attribute 'end_at_step'

Also got this once:

Traceback (most recent call last):
  File "D:\Users\pmccu\stable-diffusion-webui\modules\call_queue.py", line 57, in f
    res = list(func(*args, **kwargs))
  File "D:\Users\pmccu\stable-diffusion-webui\modules\call_queue.py", line 36, in f
    res = func(*args, **kwargs)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\txt2img.py", line 55, in txt2img
    processed = processing.process_images(p)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 732, in process_images
    res = process_images_inner(p)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
    return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 856, in process_images_inner
    p.setup_conds()
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 1309, in setup_conds
    super().setup_conds()
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 470, in setup_conds
    self.c = self.get_conds_with_caching(prompt_parser.get_multicond_learned_conditioning, prompts, total_steps, [self.cached_c], self.extra_network_data)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\processing.py", line 455, in get_conds_with_caching
    cache[1] = function(shared.sd_model, required_prompts, steps, hires_steps, shared.opts.use_old_scheduling)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\hijacker.py", line 12, in wrapper
    return function(*args, **kwargs, original_function=self.__original_functions[attribute])
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 92, in _hijacked_get_multicond_learned_conditioning
    res = original_function(*args, **kwargs)
  File "D:\Users\pmccu\stable-diffusion-webui\modules\prompt_parser.py", line 262, in get_multicond_learned_conditioning
    learned_conditioning = get_learned_conditioning(model, prompt_flat_list, steps, hires_steps, use_old_scheduling)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\hijacker.py", line 12, in wrapper
    return function(*args, **kwargs, original_function=self.__original_functions[attribute])
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 70, in _hijacked_get_learned_conditioning
    schedules = [_sample_tensor_schedules(cond_tensor, real_total_steps, is_hires=hires_steps is not None)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 70, in <listcomp>
    schedules = [_sample_tensor_schedules(cond_tensor, real_total_steps, is_hires=hires_steps is not None)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\scripts\promptlang.py", line 66, in <genexpr>
    cond_tensors = (tensor_builder.build(flattened_schedules[begin:end], empty_cond.get())
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 111, in build
    conds = self.__resize_uniformly(conds, max_cond_size, empty_cond)
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 125, in __resize_uniformly
    conds[:] = ([self.__resize_schedule(schedule, max_cond_size, empty_cond) for schedule in schedules]
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 125, in <genexpr>
    conds[:] = ([self.__resize_schedule(schedule, max_cond_size, empty_cond) for schedule in schedules]
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 125, in <listcomp>
    conds[:] = ([self.__resize_schedule(schedule, max_cond_size, empty_cond) for schedule in schedules]
  File "D:\Users\pmccu\stable-diffusion-webui\extensions\sd-prompt-fusion-extension\lib_prompt_fusion\interpolation_tensor.py", line 134, in __resize_schedule
    resized_cond = torch.concatenate([schedule.cond] + [empty_cond] * cond_missing_size)
TypeError: expected Tensor as element 0 in argument 0, but got TensorCondWrapper
ljleb commented 11 months ago

I was able to fix the second error, however I am not able to reproduce the first. Can you share the steps to reproduce the first error related to adetailer?

ljleb commented 11 months ago

If you are also not able to reproduce anymore with the new changes, then maybe the first error was caused by the second?

pmcculler commented 11 months ago

Think I found it; including a construct like this one in the prompt seems to trigger it every time.

ljleb commented 11 months ago

I am unable reproduce this issue on the latest version using the example prompt above unfortunately.

If you could share the generation parameters with prompt fusion and without adetailer, and then with adetailer but without prompt fusion, so that I can put them together, that would be very helpful.

pmcculler commented 11 months ago

With ADetailer off and PF on, the attached params file with a leading X fails, the other one succeeds, and the only difference, I believe, is the inclusion of the statement [[(equationcore:1.1):(viking):0.0,0.3]:(crowcore):0.0,0.4]

00175-2415305156.txt X00175-2415305156.txt

pmcculler commented 11 months ago

With ADetailer on and PF off, these two, pretty much same as before, including and not include [[(equationcore:1.1):(viking):0.0,0.3]:(crowcore):0.0,0.4], both succeed.

00181-698865600.txt 00180-698865599.txt

Hope these are helpful!

pmcculler commented 11 months ago

I'm trying these again after pulling the changes made in the last few minutes. Crossing fingers! Did I mention that I love this extension so much :)

pmcculler commented 11 months ago

It no longer reproduces with the very latest code. Woot! Thank you! Now it separately appears that ADetailer has some kind of problem... :)

ljleb commented 11 months ago

Nice! I'll close this then.

I'm glad you like this repo! Feel free to suggest changes or PRs any time if you have a need.