[Bug]: Outputs very pixelated / low bitrate looking when img2img is used on sketch

[ghost]

What happened?

I thought about using openOutpaint as a sketch editor to first paint a white surface and then paint a crude hammer on it (pictures below). I then intended to use img2img on it to get an object that can be cut out later, but the outputs are very pixelated. "Affect mask" was turned off. Inpainting model selected. Denoise was too high (didnt hold the form) but lower denoise didnt change the pixelated appearance.

I also noticed that if I use txt2img on an empty surface with prompt+neg. prompt the results are... really strange, like this:

Its as if openOutpaint has problems processing against the empty background, even if it is painted over

Steps to reproduce the problem

Its in the description

What should have happened?

usual renders

Commit where the problem happens

newest

What platforms do you use to access openOutpaint?

Windows

What browsers do you use to access the UI ?

Google Chrome

Browser Extensions/Addons

None that would be relevant. Also tried running in private window which didnt change it

AUTOMATIC1111 webUI Commandline Arguments

--xformers --api

Additional information

No response

[zero01101]

i kind of suspect this might be related to the model you're using as that's definitely a supported use case

I also noticed that if I use txt2img on an empty surface with prompt+neg. prompt the results are... really strange, like this:

txt2img meaning the dream tool?

openOutpaint's default parameters definitely use both positive and negative prompts and both work as expected; this "test image" of mine has proven to be a very reliable test of determinism

your image really feels like your issue is going to be the model you're using or possibly you've got resolution and reticle size set... really mismatched?

do you get those results using runwayML 1.5 or stabilityAI 2.0 inpainting models using 512x512 matched resolution/reticle?

[zero01101]

oh! regarding the txt2img issues, you don't possibly have hrfix enabled with a high scale factor, do you? from the lie to HRfix section of the manual:

A 512x512 reticle with the HRfix scale factor mistakenly left on 4 will send the API a request for a 128x128 image, upscaled 4x. This will almost undoubtedly look horrible.

[ghost]

Hm, no I dont have it on 4. And the SD model is the usual 1.5 inpainting model, I dont know where I got it from, but I also included the VAE. It gives me very good results but only when I import something and then start to outpaint the borders. If I start empty its problematic

Im going to test it with another browser or re-download the model with the VAE.

[ghost]

Yea I found the error. I unchecked apply txt2img Hrfix and it works now. I was under the impression that txt2img hrfix is disabled by default if I only use 512x512. That explains everything!