search

invoke-ai / InvokeAI

InvokeAI is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, supports terminal use through a CLI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
22.39k stars 2.32k forks source link

[bug]: Upscaling fails with error "TypeError: expected string or bytes-like object" #1956

Closed alimony closed 1 year ago

alimony commented 1 year ago

Is there an existing issue for this?

OS

macOS

GPU

mps

VRAM

16 GB

What happened?

When I run upscaling using the !fix command, invoke.py crashes. This is the input:

> !fix /Users/markus/invokeai/outputs/000001.4033572308.png -U 4 1

This is the output:

>> fixing /Users/markus/invokeai/outputs/000001.4033572308.png
>> using seed 4033572308 and prompt "nature illustration" for /Users/markus/invokeai/outputs/000001.4033572308.png
>> Parsed prompt to FlattenedPrompt:[Fragment:'nature illustration'@1.0]
>> Parsed negative prompt to FlattenedPrompt:[Fragment:''@1.0]
/Users/markus/Documents/InvokeAI/ldm/modules/embedding_manager.py:166: UserWarning: The operator 'aten::nonzero' is not  currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at  /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:11.)
  placeholder_rows, placeholder_cols = torch.where(

>> Tokens (prompt) (4):
nature illustration 

>> Tokens (unconditioning) (0):

>> Real-ESRGAN Upscaling seed:4033572308 : scale:4x
    Tile 1/4
    Tile 2/4
    Tile 3/4
    Tile 4/4
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/markus/Documents/InvokeAI/scripts/invoke.py:4 in <module>                                 │
│                                                                                                  │
│   1 #!/usr/bin/env python                                                                        │
│   2                                                                                              │
│   3 import ldm.invoke.CLI                                                                        │
│ ❱ 4 ldm.invoke.CLI.main()                                                                        │
│   5                                                                                              │
│   6                                                                                              │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/invoke/CLI.py:133 in main                                   │
│                                                                                                  │
│   130 │   │   )                                                                                  │
│   131 │                                                                                          │
│   132 │   try:                                                                                   │
│ ❱ 133 │   │   main_loop(gen, opt)                                                                │
│   134 │   except KeyboardInterrupt:                                                              │
│   135 │   │   print("\ngoodbye!")                                                                │
│   136                                                                                            │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/invoke/CLI.py:371 in main_loop                              │
│                                                                                                  │
│   368 │   │   │   │   │   print(f'** {str(e)} **')                                               │
│   369 │   │   │   elif operation == 'postprocess':                                               │
│   370 │   │   │   │   print(f'>> fixing {opt.prompt}')                                           │
│ ❱ 371 │   │   │   │   opt.last_operation = do_postprocess(gen,opt,image_writer)                  │
│   372 │   │   │                                                                                  │
│   373 │   │   │   elif operation == 'mask':                                                      │
│   374 │   │   │   │   print(f'>> generating masks from {opt.prompt}')                            │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/invoke/CLI.py:676 in do_postprocess                         │
│                                                                                                  │
│   673 │   opt.save_original  = True # do not overwrite old image!                                │
│   674 │   opt.last_operation = f'postprocess:{tool}'                                             │
│   675 │   try:                                                                                   │
│ ❱ 676 │   │   gen.apply_postprocessor(                                                           │
│   677 │   │   │   image_path      = file_path,                                                   │
│   678 │   │   │   tool            = tool,                                                        │
│   679 │   │   │   facetool_strength = opt.facetool_strength,                                     │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/generate.py:628 in apply_postprocessor                      │
│                                                                                                  │
│    625 │   │   │   elif tool == 'upscale':                                                       │
│    626 │   │   │   │   facetool = 'gfpgan'   # but won't be run                                  │
│    627 │   │   │   │   facetool_strength = 0                                                     │
│ ❱  628 │   │   │   return self.upscale_and_reconstruct(                                          │
│    629 │   │   │   │   [[image,seed]],                                                           │
│    630 │   │   │   │   facetool = facetool,                                                      │
│    631 │   │   │   │   strength = facetool_strength,                                             │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/generate.py:944 in upscale_and_reconstruct                  │
│                                                                                                  │
│    941 │   │   │   │   )                                                                         │
│    942 │   │   │                                                                                 │
│    943 │   │   │   if image_callback is not None:                                                │
│ ❱  944 │   │   │   │   image_callback(image, seed, upscaled=True, use_prefix=prefix)             │
│    945 │   │   │   else:                                                                         │
│    946 │   │   │   │   r[0] = image                                                              │
│    947                                                                                           │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/invoke/CLI.py:313 in image_writer                           │
│                                                                                                  │
│   310 │   │   │   │   │   if use_prefix is not None:                                             │
│   311 │   │   │   │   │   │   prefix = use_prefix                                                │
│   312 │   │   │   │   │   postprocessed = upscaled if upscaled else operation=='postprocess'     │
│ ❱ 313 │   │   │   │   │   opt.prompt = gen.concept_lib().replace_triggers_with_concepts(opt.pr   │
│   314 │   │   │   │   │   filename, formatted_dream_prompt = prepare_image_metadata(             │
│   315 │   │   │   │   │   │   opt,                                                               │
│   316 │   │   │   │   │   │   prefix,                                                            │
│                                                                                                  │
│ /Users/markus/Documents/InvokeAI/ldm/invoke/concepts_lib.py:87 in                                │
│ replace_triggers_with_concepts                                                                   │
│                                                                                                  │
│    84 │   │   better to store the concept name (unique) than the concept trigger                 │
│    85 │   │   (not necessarily unique!)                                                          │
│    86 │   │   '''                                                                                │
│ ❱  87 │   │   triggers = self.match_trigger.findall(prompt)                                      │
│    88 │   │   if not triggers:                                                                   │
│    89 │   │   │   return prompt                                                                  │
│    90                                                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: expected string or bytes-like object

Screenshots

No response

Additional context

I'm using the latest source checkout (https://github.com/invoke-ai/InvokeAI/commit/36e6908266996255511abcf2d21469164fd66c34). It seems like this is related to https://github.com/invoke-ai/InvokeAI/issues/1911 and https://github.com/invoke-ai/InvokeAI/issues/1647

Contact Details

No response

lstein commented 1 year ago

Thank you. I'll track this down and get it fixed ASAP.

lstein commented 1 year ago

I've just sent in a pull request with the fix. When this is merged in (probably sometime this evening), you can upgrade by running update.sh in the runtime directory (if you used the installer), or git pull in the repository if you used a manual method.