search

Sanster / IOPaint

Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
https://www.iopaint.com/
Apache License 2.0
17.35k stars 1.75k forks source link

[BUG] GFPGAN error when GFPGAN is installed #470

Open SoftologyPro opened 2 months ago

SoftologyPro commented 2 months ago

Using this command

iopaint start --model=lama --device=cuda --port=8080 --enable-interactive-seg --interactive-seg-model=vit_l --interactive-seg-device=cuda --enable-anime-seg --enable-gfpgan --gfpgan-device cuda --enable-remove-bg

gives this error

2024-03-04 09:45:19.135 | INFO     | iopaint.runtime:setup_model_dir:82 - Model directory: C:\Users\Jason\.cache
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.8
- torch: 2.2.0+cu118
- torchvision: 0.17.0+cu118
- Pillow: 9.5.0
- diffusers: 0.26.3
- transformers: 4.38.2
- opencv-python: 4.9.0.80
- accelerate: 0.27.2
- iopaint: 1.2.1
- rembg: 2.0.55
- realesrgan: 0.3.0
- gfpgan: 1.3.8

[W init.cpp:768] Warning: nvfuser is no longer supported in torch script, use _jit_set_nvfuser_enabled is deprecated and a no-op (function operator ())
2024-03-04 09:45:21.097 | INFO     | iopaint.plugins:build_plugins:32 - Initialize InteractiveSeg plugin
2024-03-04 09:45:21.097 | INFO     | iopaint.plugins.interactive_seg:_init_session:62 - SegmentAnything model path: C:\Users\Jason\.cache\torch\hub\checkpoints\sam_vit_l_0b3195.pth
2024-03-04 09:45:22.858 | INFO     | iopaint.plugins:build_plugins:38 - Initialize RemoveBG plugin
2024-03-04 09:45:23.902 | INFO     | iopaint.plugins:build_plugins:42 - Initialize AnimeSeg plugin
2024-03-04 09:45:24.008 | INFO     | iopaint.helper:load_model:123 - Loading model from: C:\Users\Jason\.cache\torch\hub\checkpoints\isnetis.pth
2024-03-04 09:45:24.104 | INFO     | iopaint.plugins:build_plugins:56 - Initialize GFPGAN plugin
2024-03-04 09:45:24.104 | INFO     | iopaint.plugins:build_plugins:60 - GFPGAN no background upscaler, use --enable-realesrgan to enable it
2024-03-04 09:45:24.281 | ERROR    | iopaint.plugins.base_plugin:__init__:15 - gfpgan is not installed, please install it first. pip install gfpgan
{
    "host": "127.0.0.1",
    "port": 8080,
    "inbrowser": false,
    "model": "lama",
    "no_half": false,
    "low_mem": false,
    "cpu_offload": false,
    "disable_nsfw_checker": false,
    "local_files_only": false,
    "cpu_textencoder": false,
    "device": "cuda",
    "input": null,
    "output_dir": null,
    "quality": 95,
    "enable_interactive_seg": true,
    "interactive_seg_model": "vit_l",
    "interactive_seg_device": "cuda",
    "enable_remove_bg": true,
    "remove_bg_model": "briaai/RMBG-1.4",
    "enable_anime_seg": true,
    "enable_realesrgan": false,
    "realesrgan_device": "cpu",
    "realesrgan_model": "realesr-general-x4v3",
    "enable_gfpgan": true,
    "gfpgan_device": "cuda",
    "enable_restoreformer": false,
    "restoreformer_device": "cpu"
}

You can see gfpgan is installed as it shows it is version 1.3.8. GFPGAN was installed with pip install gfpgan

What am I missing?

SoftologyPro commented 2 months ago

Similar error for realesrgan. I lauinch with this command

iopaint start --model=lama --device=cuda --port=8080 --enable-interactive-seg --interactive-seg-model=vit_l --interactive-seg-device=cuda --enable-anime-seg --enable-realesrgan --realesrgan-model RealESRGAN_x4plus --realesrgan-device cuda --enable-remove-bg

and get this error (you can see realesrgan is detected as installed)

2024-03-04 09:49:37.458 | INFO     | iopaint.runtime:setup_model_dir:82 - Model directory: C:\Users\Jason\.cache
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.8
- torch: 2.2.0+cu118
- torchvision: 0.17.0+cu118
- Pillow: 9.5.0
- diffusers: 0.26.3
- transformers: 4.38.2
- opencv-python: 4.9.0.80
- accelerate: 0.27.2
- iopaint: 1.2.1
- rembg: 2.0.55
- realesrgan: 0.3.0
- gfpgan: 1.3.8

[W init.cpp:768] Warning: nvfuser is no longer supported in torch script, use _jit_set_nvfuser_enabled is deprecated and a no-op (function operator ())
2024-03-04 09:49:39.317 | INFO     | iopaint.plugins:build_plugins:32 - Initialize InteractiveSeg plugin
2024-03-04 09:49:39.317 | INFO     | iopaint.plugins.interactive_seg:_init_session:62 - SegmentAnything model path: C:\Users\Jason\.cache\torch\hub\checkpoints\sam_vit_l_0b3195.pth
2024-03-04 09:49:41.083 | INFO     | iopaint.plugins:build_plugins:38 - Initialize RemoveBG plugin
2024-03-04 09:49:42.154 | INFO     | iopaint.plugins:build_plugins:42 - Initialize AnimeSeg plugin
2024-03-04 09:49:42.261 | INFO     | iopaint.helper:load_model:123 - Loading model from: C:\Users\Jason\.cache\torch\hub\checkpoints\isnetis.pth
2024-03-04 09:49:42.359 | INFO     | iopaint.plugins:build_plugins:46 - Initialize RealESRGAN plugin: RealESRGAN_x4plus, cuda
2024-03-04 09:49:42.527 | ERROR    | iopaint.plugins.base_plugin:__init__:15 - RealESRGAN is not installed, please install it first. pip install realesrgan
{
    "host": "127.0.0.1",
    "port": 8080,
    "inbrowser": false,
    "model": "lama",
    "no_half": false,
    "low_mem": false,
    "cpu_offload": false,
    "disable_nsfw_checker": false,
    "local_files_only": false,
    "cpu_textencoder": false,
    "device": "cuda",
    "input": null,
    "output_dir": null,
    "quality": 95,
    "enable_interactive_seg": true,
    "interactive_seg_model": "vit_l",
    "interactive_seg_device": "cuda",
    "enable_remove_bg": true,
    "remove_bg_model": "briaai/RMBG-1.4",
    "enable_anime_seg": true,
    "enable_realesrgan": true,
    "realesrgan_device": "cuda",
    "realesrgan_model": "RealESRGAN_x4plus",
    "enable_gfpgan": false,
    "gfpgan_device": "cpu",
    "enable_restoreformer": false,
    "restoreformer_device": "cpu"
}
SoftologyPro commented 2 months ago

Also same if I try restoreformer (but this complaiins about GFPGAN again)

iopaint start --model=lama --device=cuda --port=8080 --enable-interactive-seg --interactive-seg-model=vit_l --interactive-seg-device=cuda --enable-anime-seg --enable-remove-bg --enable-restoreformer --restoreformer-device cuda

error is

2024-03-04 09:51:55.078 | INFO     | iopaint.runtime:setup_model_dir:82 - Model directory: C:\Users\Jason\.cache
- Platform: Windows-10-10.0.22621-SP0
- Python version: 3.10.8
- torch: 2.2.0+cu118
- torchvision: 0.17.0+cu118
- Pillow: 9.5.0
- diffusers: 0.26.3
- transformers: 4.38.2
- opencv-python: 4.9.0.80
- accelerate: 0.27.2
- iopaint: 1.2.1
- rembg: 2.0.55
- realesrgan: 0.3.0
- gfpgan: 1.3.8

[W init.cpp:768] Warning: nvfuser is no longer supported in torch script, use _jit_set_nvfuser_enabled is deprecated and a no-op (function operator ())
2024-03-04 09:51:56.956 | INFO     | iopaint.plugins:build_plugins:32 - Initialize InteractiveSeg plugin
2024-03-04 09:51:56.956 | INFO     | iopaint.plugins.interactive_seg:_init_session:62 - SegmentAnything model path: C:\Users\Jason\.cache\torch\hub\checkpoints\sam_vit_l_0b3195.pth
2024-03-04 09:51:58.679 | INFO     | iopaint.plugins:build_plugins:38 - Initialize RemoveBG plugin
2024-03-04 09:52:00.270 | INFO     | iopaint.plugins:build_plugins:42 - Initialize AnimeSeg plugin
2024-03-04 09:52:00.386 | INFO     | iopaint.helper:load_model:123 - Loading model from: C:\Users\Jason\.cache\torch\hub\checkpoints\isnetis.pth
2024-03-04 09:52:00.489 | INFO     | iopaint.plugins:build_plugins:69 - Initialize RestoreFormer plugin
2024-03-04 09:52:00.662 | ERROR    | iopaint.plugins.base_plugin:__init__:15 - gfpgan is not installed, please install it first. pip install gfpgan
{
    "host": "127.0.0.1",
    "port": 8080,
    "inbrowser": false,
    "model": "lama",
    "no_half": false,
    "low_mem": false,
    "cpu_offload": false,
    "disable_nsfw_checker": false,
    "local_files_only": false,
    "cpu_textencoder": false,
    "device": "cuda",
    "input": null,
    "output_dir": null,
    "quality": 95,
    "enable_interactive_seg": true,
    "interactive_seg_model": "vit_l",
    "interactive_seg_device": "cuda",
    "enable_remove_bg": true,
    "remove_bg_model": "briaai/RMBG-1.4",
    "enable_anime_seg": true,
    "enable_realesrgan": false,
    "realesrgan_device": "cpu",
    "realesrgan_model": "realesr-general-x4v3",
    "enable_gfpgan": false,
    "gfpgan_device": "cpu",
    "enable_restoreformer": true,
    "restoreformer_device": "cuda"
}
SoftologyPro commented 2 months ago

If I remove those 3 parameters and launch with

iopaint start --model=lama --device=cuda --port=8080 --enable-interactive-seg --interactive-seg-model=vit_l --interactive-seg-device=cuda --enable-anime-seg --enable-remove-bg

It starts and runs fine.

Sanster commented 2 months ago

The logs of iopaint did not print the complete error message. Can you try manually importing import gfpgan and import realesrgan to see what error occurs?

SoftologyPro commented 2 months ago

The logs of iopaint did not print the complete error message. Can you try manually importing import gfpgan and import realesrgan to see what error occurs?

How do I add those inport lines? I setup IOPaint with pip install iopaint, so there is no code to modify.

Specificaly, I setup a new clean environment and then install the requirements with

python -m pip install --upgrade pip
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts wheel==0.42.0
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts rembg==2.0.55
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts realesrgan==0.3.0
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts gfpgan==1.3.8
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts iopaint==1.2.1
pip uninstall -y torch
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts torch==2.2.0+cu118 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip uninstall -y charset-normalizer
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts charset-normalizer==3.3.2
pip uninstall -y typing_extensions
pip install --no-cache-dir --ignore-installed --force-reinstall --no-warn-conflicts typing_extensions==4.8.0
Sanster commented 2 months ago

Create a python file, name it whatever you want, for example tmp.py, with only one line of content:

import gfpgan

Then execute python tmp.py, this command should produce an error, please paste the error message.

SoftologyPro commented 2 months ago

ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

Does it need a torch version older than 2.2.0?

OK, fixed, it needs Torch 2.1.0 to work. Maybe specify the exact version in your requirements. https://softologyblog.wordpress.com/2023/10/10/a-plea-to-all-python-developers/

RealESRGAN and RestoreFormer work fine now too.

Thanks for your help.

Sanster commented 2 months ago

ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

Does it need a torch version older than 2.2.0?

OK, fixed, it needs Torch 2.1.0 to work. Maybe specify the exact version in your requirements. https://softologyblog.wordpress.com/2023/10/10/a-plea-to-all-python-developers/

RealESRGAN and RestoreFormer work fine now too.

Thank you for your feedback. It seems that there have been changes in the api due to torchvision upgrades. I think I can refer to InvokeAI's method https://github.com/invoke-ai/InvokeAI/pull/5690 and add all the dependent code to IOPaint.

carlxwz commented 1 month ago

edit the file: /usr/local/lib/python3.10/dist-packages/basicsr/data/degradations.py in line 8 change: from torchvision.transforms.functional_tensor import rgb_to_grayscale to: from torchvision.transforms._functional_tensor import rgb_to_grayscale Because in Torch 2.2.0functional_tensor rename to _functional_tensor

jmaple12 commented 1 month ago

edit the file: /usr/local/lib/python3.10/dist-packages/basicsr/data/degradations.py in line 8 change: from torchvision.transforms.functional_tensor import rgb_to_grayscale to: from torchvision.transforms._functional_tensor import rgb_to_grayscale Because in Torch 2.2.0functional_tensor rename to _functional_tensor

thanks，I find this file on "D:\Anaconda\Lib\site-packages\basicsr\data\degradations.py",