search

invoke-ai / InvokeAI

Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.
https://invoke-ai.github.io/InvokeAI/
Apache License 2.0
23.84k stars 2.45k forks source link

[bug]: outpainting and inpainting It gives a very bad result v4.0rc2 #5972

Closed wolfkingal2000 closed 7 months ago

wolfkingal2000 commented 8 months ago

Is there an existing issue for this problem?

Operating system

Linux

GPU vendor

Nvidia (CUDA)

GPU model

No response

GPU VRAM

15

Version number

4.0rc2

Browser

Opera GX Browser

Python dependencies

[2024-03-15 20:54:08,268]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "POST /api/v1/images/upload?image_category=general&is_intermediate=true HTTP/1.1" 201 [2024-03-15 20:54:09,284]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "POST /api/v1/images/upload?image_category=mask&is_intermediate=true HTTP/1.1" 201 [2024-03-15 20:54:10,093]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "POST /api/v1/queue/default/enqueue_batch HTTP/1.1" 200 [2024-03-15 20:54:10,725]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/queue/default/status HTTP/1.1" 200 4% 1/26 [00:00<00:08, 2.86it/s][2024-03-15 20:54:11,502]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/16fbb6b5-b922-4f97-80ab-1b175806c7da.png HTTP/1.1" 200 8% 2/26 [00:00<00:08, 2.86it/s][2024-03-15 20:54:12,124]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/1e1c99bd-6529-4ff0-a59b-748f5ba15645.png HTTP/1.1" 200 23% 6/26 [00:02<00:07, 2.79it/s][2024-03-15 20:54:13,557]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/baa295c7-cfe0-4f71-b5ce-26bbaf194bab.png HTTP/1.1" 200 100% 26/26 [00:09<00:00, 2.79it/s] [2024-03-15 20:54:22,196]::[InvokeAI]::INFO --> Graph stats: a4850d08-3a3a-48df-ae2c-873a053e51d5 Node Calls Seconds VRAM Used main_model_loader 1 0.001s 3.201G clip_skip 1 0.000s 3.201G compel 2 0.001s 3.201G noise 1 0.001s 3.201G img_resize 4 0.423s 3.201G create_gradient_mask 1 0.028s 3.201G vae_loader 1 0.000s 3.201G i2l 1 0.582s 4.476G denoise_latents 1 9.441s 3.462G l2i 1 1.285s 5.217G canvas_paste_back 1 0.280s 3.181G TOTAL GRAPH EXECUTION TIME: 12.041s TOTAL GRAPH WALL TIME: 12.054s RAM used by InvokeAI process: 8.24G (+0.000G) RAM used to load models: 1.76G VRAM in use: 3.181G RAM cache statistics: Model cache hits: 4 Model cache misses: 0 Models cached: 6 Models cleared from cache: 0 Cache high water mark: 2.74/4.00G

[2024-03-15 20:54:23,498]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/3f7e56a5-630d-4dda-a9bd-c2711cebd012.png HTTP/1.1" 200 [2024-03-15 20:54:23,508]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/fbe9f47f-8e97-4119-a006-3746f9d0a090.png HTTP/1.1" 200 [2024-03-15 20:54:24,859]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/fa109ee5-5dd0-4c5a-8e5a-61779c9bfa63.png HTTP/1.1" 200 [2024-03-15 20:54:27,965]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/images/i/fbe9f47f-8e97-4119-a006-3746f9d0a090.png/full HTTP/1.1" 200 [2024-03-15 20:54:53,100]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "PUT /api/v1/queue/default/cancel_by_batch_ids HTTP/1.1" 200 [2024-03-15 20:54:55,770]::[uvicorn.access]::INFO --> 23.150.248.50:0 - "GET /api/v1/queue/default/status HTTP/1.1" 200

What happened

outpainting opera_ZCBTFRbdpr inpainting opera_S8w5pGPbxW

and this my setting and abort opera_7s3Dh9pSNX opera_c5U9ecmmTO opera_vGHznNrBLM

What you expected to happen

.

How to reproduce the problem

No response

Additional context

No response

Discord username

No response

psychedelicious commented 8 months ago

Hi, two issues:

  1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
  2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?
wolfkingal2000 commented 8 months ago

Hi, two issues:

  1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
  2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

I'm using Colab for installation so use git clone, and for Torch, this command fixes the issue: !pip install --force-reinstall torch==2.1.2 --index-url https://download.pytorch.org/whl/cu121 !pip install -U typing-extensions !pip install -U fsspec==2023.5.0 !pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121

Also, there's something else: "invokeai-model-install.py." I've checked the latest version, but it seems to have been removed. can restored back Since I use Colab, it was very useful. Could you provide any guidance on using new [v4.0.0rc4] installation i installed but dont find invokeai.yaml file ? Thank you.

and for [v4.0.0rc4] when run invokeai-web.py give this error can tell me what is problem

Traceback (most recent call last): File "/usr/local/bin/invokeai-web", line 8, in sys.exit(run_app()) File "/content/InvokeAI/invokeai/app/run_app.py", line 10, in run_app from invokeai.app.api_app import invoke_api File "/content/InvokeAI/invokeai/app/api_app.py", line 19, in from torch.backends.mps import is_available as is_mps_available File "/usr/local/lib/python3.10/dist-packages/torch/init.py", line 237, in from torch._C import * # noqa: F403 ImportError: /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cuda.so: undefined symbol: ncclCommRegister http (80)

psychedelicious commented 8 months ago

Ok, we can't do anything to help with colab. I just answered a similar question from your other issue.

Let's keep this issue focused to the inpainting issue.

wolfkingal2000 commented 8 months ago

Ok, we can't do anything to help with colab. I just answered a similar question from your other issue.

Let's keep this issue focused to the inpainting issue.

It's ok, but I had a question, can you tell me all the prerequisites of the program, maybe I can fix it myself with the required versions, because the problem started with version 4.0.0rc4, 4.0.0rc2 ran without any problems and did not give this error.

psychedelicious commented 8 months ago

You can review https://github.com/invoke-ai/InvokeAI/blob/main/pyproject.toml to see the list of dependencies. Note you probably need the xformers group of optional dependencies too.

wolfkingal2000 commented 8 months ago

You can review https://github.com/invoke-ai/InvokeAI/blob/main/pyproject.toml to see the list of dependencies. Note you probably need the xformers group of optional dependencies too.

i fix that problem thanks

but get this error for use in painting model, i use , in v4.0.0rc5

[2024-03-27 19:45:36,699]::[InvokeAI]::INFO --> Graph stats: 7a192474-3be8-41ae-bcb5-4832420f13d7 Node Calls Seconds VRAM Used main_model_loader 1 0.001s 0.000G clip_skip 1 0.000s 0.000G compel 1 0.597s 0.000G TOTAL GRAPH EXECUTION TIME: 0.598s TOTAL GRAPH WALL TIME: 0.600s RAM used by InvokeAI process: 1.28G (+0.030G) RAM used to load models: 0.00G RAM cache statistics: Model cache hits: 0 Model cache misses: 0 Models cached: 0 Models cleared from cache: 0 Cache high water mark: 0.00/0.00G

[2024-03-27 19:45:38,510]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "POST /api/v1/images/upload?image_category=general&is_intermediate=true HTTP/1.1" 201 [2024-03-27 19:45:39,119]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "GET /api/v1/queue/default/status HTTP/1.1" 200 [2024-03-27 19:45:39,255]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "POST /api/v1/images/upload?image_category=mask&is_intermediate=true HTTP/1.1" 201 [2024-03-27 19:45:41,029]::[uvicorn.access]::INFO --> 67.220.86.150:0 - "POST /api/v1/queue/default/enqueue_batch HTTP/1.1" 200 [2024-03-27 19:45:41,085]::[ModelLoadService]::INFO --> Converting /root/invokeai/models/sd-1/main/meinamix_v11-inpainting.safetensors to diffusers format [2024-03-27 19:45:41,224]::[InvokeAI]::ERROR --> Error while invoking session b389a36a-3695-4c99-800b-9f53f78945a6, invocation 487f135b-e0ef-4fad-9e1f-64e27c9fe533 (compel): Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect. [2024-03-27 19:45:41,224]::[InvokeAI]::ERROR --> Traceback (most recent call last): File "/content/InvokeAI/invokeai/app/services/session_processor/session_processor_default.py", line 160, in _process outputs = self._invocation.invoke_internal( File "/content/InvokeAI/invokeai/app/invocations/baseinvocation.py", line 281, in invoke_internal output = self.invoke(context) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/content/InvokeAI/invokeai/app/invocations/compel.py", line 57, in invoke tokenizer_info = context.models.load(self.clip.tokenizer) File "/content/InvokeAI/invokeai/app/services/shared/invocation_context.py", line 339, in load return self._services.model_manager.load.load_model(model, _submodel_type, self._data) File "/content/InvokeAI/invokeai/app/services/model_load/model_load_default.py", line 80, in load_model ).load_model(model_config, submodel_type) File "/content/InvokeAI/invokeai/backend/model_manager/load/load_default.py", line 62, in load_model model_path = self._convert_if_needed(model_config, model_path, submodel_type) File "/content/InvokeAI/invokeai/backend/model_manager/load/load_default.py", line 82, in _convert_if_needed return self._convert_model(config, model_path, cache_path) File "/content/InvokeAI/invokeai/backend/model_manager/load/model_loaders/stable_diffusion.py", line 90, in _convert_model convert_ckpt_to_diffusers( File "/content/InvokeAI/invokeai/backend/model_manager/convert_ckpt_to_diffusers.py", line 46, in convert_ckpt_to_diffusers pipe = download_from_original_stable_diffusion_ckpt(Path(checkpoint_path).as_posix(), kwargs) File "/usr/local/lib/python3.10/dist-packages/diffusers/pipelines/stable_diffusion/convert_from_ckpt.py", line 1472, in download_from_original_stable_diffusion_ckpt set_module_tensor_to_device(unet, param_name, "cpu", value=param) File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/modeling.py", line 348, in set_module_tensor_to_device raise ValueError( ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.

psychedelicious commented 8 months ago

ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.

We just fixed that in #6064 , will be in the next release

wolfkingal2000 commented 8 months ago

ValueError: Trying to set a tensor of shape torch.Size([320, 9, 3, 3]) in "weight" (which has shape torch.Size([320, 4, 3, 3])), this look incorrect.

We just fixed that in #6064 , will be in the next release

Thanks and add multi links in query in model manger use to add a lot models together

psychedelicious commented 8 months ago

Please create a separate issue for each feature request or bug report.

wolfkingal2000 commented 8 months ago

Hi, two issues:

  1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
  2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

hi vr 4.0.0 rc6 out painting and in painting They are still bad i down vr torch but dont fixed that

opera_LDeO4tWN9A

this my setting accelerate 0.28.0 compel 2.0.2 cuda 12.1 diffusers 0.27.2 numpy 1.26.4 opencv 4.9.0.80 onnx 1.15.0 pillow 9.4.0 python 3.10.12 torch 2.1.2+cu121 torchvision 0.16.2+cu121 transformers 4.39.1 xformers Not Installed

wolfkingal2000 commented 8 months ago

Hi, two issues:

  1. We found an issue with torch 2.2.1 and inpainting models. We'll either fix or roll back the version.
  2. You have mismatched torch versions. When you created this issue, torch was pinned to 2.1. How did you install the RC? Did you manually install anything?

I think found the problem in the conversion safetensors or ckpt to diffusers model Inpainting make bad result and use diffusers like dreamshaper-8-inpainting from https://huggingface.co don't make bad result and for safetensors you can test https://civitai.com/api/download/models/120702 this