Inpainting: significant drop in resolution

vredesbyyrd commented 8 months ago

First, thank you for sharing your work and the detailed install/usage instructions.

I am a complete noob when it comes to using any AI-assisted workflows. I figured it way past time that I try and educate myself. I am mainly interested with the inpainting plugin, as it could dramatically speed up my workflow and allow for things not possible before.

When following the directions in the doc, the inpainting "works", but the entire image becomes very pixelated. Am I understanding correctly that a 512x512 layer is basically stretched across the entire image? Is there any way to workaround this? Or am I overlooking something obvious?

Apologies if this is naive. I am currently reading into "generated-fill" and trying to get a grasp on how things work.

Thanks for your time!

EDIT: For clarity, I am using the openvino inpainting plugin

gblong1 commented 8 months ago

Hi @vredesbyyrd - Do you have an example (screen shots)? That would help me understand the issue a better.

vredesbyyrd commented 8 months ago

Hi, sorry for the delay.

I nuked the original install, reinstalled with the updated directions/code which appeared to go smoothly, but unfortunately ran into unrelated issue.

Issue: inpainting... is greyed out in the gimp -> layer menu.

And when starting gimp the following is logged:

Skipping unknown file '/home/clu/build/openvino-ai-plugins-gimp/gimpopenvino/plugins/__init__.py' in plug-in directory.
Skipping unknown file '/home/clu/build/openvino-ai-plugins-gimp/gimpopenvino/plugins/plugin_utils.py' in plug-in directory.

The specified paths are what install.sh logged during the install process.

For completeness, here is some basic info:

archlinux
gimp 2.99.16-2
openvino 2023.2.0-1 (compiled from aur)
Python 3.11.6

Looking at openvino release notes, python 3.11 support started with version 2023.0.0, so I don't think that is the issue?

I initially thought the python version might be the issue, so I installed python 3.10, modified install.sh to use 3.10, and recompiled openvino with...

cmake -DPYTHON_EXECUTABLE=/usr/bin/python3.10 -DPYTHON_LIBRARY=/usr/lib/libpython3.10.so -DPYTHON_INCLUDE_DIR=/usr/include/python3.10

but ultimately ran into the same issue as described above.

Lastly, the GI_TYPELIB_PATH is different on arch, so I did export GI_TYPELIB_PATH=/usr/lib/girepository-1.0

I'm out of ideas at this point, any thoughts or pointers you may have would be really appreciated :)

vredesbyyrd commented 8 months ago

EDIT: When building 2.99.16 from source InPainting is still grayed out.

Building gimp from master (2.99.17) inpainting is no longer greyed out, but the plugin fails to run, same "problem" #68. I'm only noting this as it's interesting that the plugin was at least available.

Prior to this I was using the gimp-develop package from the aur, which is on gimp 2.99.16-2. Now wondering if it's something with that package specifically. I'll try and compile it myself and cherry pick the correct commit.

arisha07 commented 8 months ago

Hi @vredesbyyrd, just wanted to make sure that you are selecting the original image layer and mask layer before attempting to select the inpainting plugin from the drop-down list. We also have stable-diffusion-inpainting enabled that you can try.

vredesbyyrd commented 8 months ago

I did make sure both layers are selected prior to attempting to select the inpainting plugin. I'll test out the stable-diffusion-inpainting as well.

gblong1 commented 7 months ago

Hi @vredesbyyrd - any additional help needed? Otherwise we can close this issue.

vredesbyyrd commented 7 months ago

It can be closed. I was not able to get the openvino inpainting to work, but Ill probably give it a go sometime down the road.

dset0x commented 6 months ago

Hi all,

First of all, thank you for this software. I believe there is no equivalent that can be had freely.

Since several issues have been brought up in here I will post here before opening any new issues.

Using Ubuntu 23.10, pyenv (python 3.11), GIMP 2.99.16 d3c5536 and l_openvino_toolkit_ubuntu22_2023.3.0.13775.ceeafaf64f3_x86_64.tgz

Issue 1: In-painting button is grayed out.

I ran into this also. If I start with my JPG and add a layer, it is not grayed out. However, if I make any adjustments (e.g. crop the image), the button becomes grayed out. The workaround I found was to change the Image -> Mode from RGB to Indexed, and then back to RGB. This is an issue as colors are lost.

Issue 2: Image resolution degrades

Attempt 1

I start with an image of 1459x2189,
Draw over this pesky trashcan,
and run in-fill,

The entire image gets replaced with a lower resolution one. The in-fill result is passable.

Attempt 2

I crop a 512x512 px square out of the image.
I perform "Layers to Image Size" and do the Indexed/RGB dance.
I run in-painting: The results are not passable (Not enough context?)

Attempt 3

I resize the image to a lower resolution so I can provide more context in the 512x512 px square.
I perform the Indexed/RGB dance (I have to do it twice for some reason) and crop out a 512x512 px square.
And run in-painting Not bad.
I paste this back into the original image and use a layer mask to hide everything except the trashcan. The results are not that great since it's very obvious that area has been retouched.

Issue 3

stable-diffusion-inpainting: I am not sure how to enable this. Have I missed a step? ("Setup done for superresolution, semantic-segmentation, style-transfer, in-painting")

Querying plug-in: '/home/user/.pyenv/versions/openvino/lib/python3.11/site-packages/gimpopenvino/plugins/stable-diffusion-ov/stable-diffusion-ov.py'
GIMP-WARNING: gimp-2.99: gimp_wire_read(): error

Terminating plug-in: '/home/user/.pyenv/versions/openvino/lib/python3.11/site-packages/gimpopenvino/plugins/stable-diffusion-ov/stable-diffusion-ov.py'

That's all I have for now. I'll be happy to receive any tips, thanks!

arisha07 commented 6 months ago

Hi all,

First of all, thank you for this software. I believe there is no equivalent that can be had freely.

Since several issues have been brought up in here I will post here before opening any new issues.

Using Ubuntu 23.10, pyenv (python 3.11), GIMP 2.99.16 d3c5536 and l_openvino_toolkit_ubuntu22_2023.3.0.13775.ceeafaf64f3_x86_64.tgz

Issue 1: In-painting button is grayed out.

I ran into this also. If I start with my JPG and add a layer, it is not grayed out. However, if I make any adjustments (e.g. crop the image), the button becomes grayed out. The workaround I found was to change the Image -> Mode from RGB to Indexed, and then back to RGB. This is an issue as colors are lost.

Issue 2: Image resolution degrades

Attempt 1

I start with an image of 1459x2189,

Draw over this pesky trashcan,

and run in-fill,

The entire image gets replaced with a lower resolution one. The in-fill result is passable.

Attempt 2

I crop a 512x512 px square out of the image.

I perform "Layers to Image Size" and do the Indexed/RGB dance.

I run in-painting: The results are not passable (Not enough context?)

Attempt 3

I resize the image to a lower resolution so I can provide more context in the 512x512 px square.

I perform the Indexed/RGB dance (I have to do it twice for some reason) and crop out a 512x512 px square.

And run in-painting Not bad.

I paste this back into the original image and use a layer mask to hide everything except the trashcan. The results are not that great since it's very obvious that area has been retouched.

Issue 3

stable-diffusion-inpainting: I am not sure how to enable this. Have I missed a step? ("Setup done for superresolution, semantic-segmentation, style-transfer, in-painting")
Querying plug-in: '/home/user/.pyenv/versions/openvino/lib/python3.11/site-packages/gimpopenvino/plugins/stable-diffusion-ov/stable-diffusion-ov.py'
GIMP-WARNING: gimp-2.99: gimp_wire_read(): error

Terminating plug-in: '/home/user/.pyenv/versions/openvino/lib/python3.11/site-packages/gimpopenvino/plugins/stable-diffusion-ov/stable-diffusion-ov.py'
That's all I have for now. I'll be happy to receive any tips, thanks!

Hi, currently the in-painting model only supports 512x512 resolution. For stable-diffusion-inpainting, did you download the model during install script ? Please refer to the latest README guide for detailed information. Also, please note that the stable-diffusion-inpainting supports only 512x512 currently. Thanks.

dset0x commented 6 months ago

Hi, thanks for the information.

For stable-diffusion-inpainting, did you download the model during install script ? Please refer to the latest README guide for detailed information.

I did. I followed that README indeed.

intel / openvino-ai-plugins-gimp