Strange behaviour in v2

[kft334]

I'm using the A1111 dev branch (up-to-date) with the same seed and freeu values, via the API. The only thing that is changing between the two images is the freeu version.

1st image is v1 (41ca4f43acd32e0dd138d99deb1e68ef4391ed2c) 2nd image is v2 (08a1734f166b87efba96f42d79a79b2850e1bea4)

Before high-res fix pass, no discernible objects appear at all. The image converges to blue noise triangles (no flowers, dog). Then during high-res fix pass, the objects eventually appear. I don't think that it's an issue with my request since the request is the same for both v1 and v2 and as far as I can tell the parameters have not changed between those two versions.

[ljleb]

It's possible that the th.cat monkey patch stops working after a webui release. Ideally I should fix this before 1.7.0

[ljleb]

Does this still happen in webui==v1.7.0-RC? I cannot reproduce there through the web API.

How did you generate these images? The PNG info is not compatible with A1111, the problem could be caused by anything at this point.

[kft334]

I've just updated to the latest dev commit and the problem still persists. The images were generated using my own SD app which uses the A1111 API. It's reproducible 100% of the time and the only thing that changes is the FreeU version.

https://github.com/ljleb/sd-webui-freeu/assets/93904307/3e8a25b3-e714-4ad1-9b6c-88849d55c5b0

[ljleb]

Thanks for the details, I see that the problem is persistent and that updating the extension to latest seems to create a conflict somewhere.

Could you share the request that is sent to the webui? I wasn't able to find a similarly offending API request. I'm still looking for a combination of options. Maybe it's a conflict with another extension?

[kft334]

Sure, here's the request. The list of extensions that I have installed is pretty short but I'll include it as well.

  "prompt": "cat on a couch",
  "negative_prompt": "Bad Anatomy, Blurry, Deformed, Disfigured, Disproportionate, Distorted, Fused, Gross, Incoherent, Malformed, Misaligned, Misshapen, Morbid, Mutated, Mutation, Mutilated, Out Of Frame, frame, framed, text, writing, border, multiple, many, warped, bent, crooked, partial, partially visible, obscured, ugly, out of frame, low resolution, blurred, warped, Overprocessed, weird proportions, badly cropped, badly framed, low detail, bad proportions, watermark, tiling, tiles, tiled, pixelated, lowres, text, error, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, badly drawn, mutation, deformed, cloned, disfigured, gross proportions",
  "steps": 20,
  "cfg_scale": 7.5,
  "width": 640,
  "height": 640,
  "seed": 2775150635,
  "denoising_strength": 0.55,
  "n_iter": 1,
  "batch_size": 1,
  "sampler_name": "DPM++ 2M Karras",
  "sampler_index": "DPM++ 2M Karras",
  "restore_faces": false,
  "tiling": false,
  "enable_hr": true,
  "hr_scale": 1.75,
  "hr_second_pass_steps": 15,
  "hr_upscaler": "Nearest",
  "alwayson_scripts": {
    "freeu": {
      "args": [
        true,
        0.0,
        1.0,
        0.0,
        1.1,
        0.6,
        0.0,
        0.5,
        0.0,
        1.0,
        1.2,
        0.4,
        0.0,
        0.5,
        0.0,
        1.0,
        1.0,
        1.0,
        0.0,
        0.5,
        0.0,
        1.0
      ]
    }
  }
}

List of installed extensions: sd-dynamic-thresholding sd-webui-animateddiff sd-webui-api-payload-display sd-webui-controlnet sd-webui-freeu sd-webui-kohya-hiresfix sd-webui-loractl

[ljleb]

I was able to reproduce your situation.

The number of parameters for the script changed in the latest commit. Because of this, since you are passing the parameters as a list, every parameter was offset. You then end up with a very bad freeu config for inference. In other words, it's not a bug, just a really bad configuration that does not give good results.

You may want to try to pass a dict instead of a list, as indicated in the readme, if you intend your code to be compatible with multiple versions of the script.

[ljleb]

If you are interested in only supporting the latest version of the extension, then the freeu version is a new parameter that has to be passed:

"alwayson_scripts": {
    "freeu": {
      "args": [
        true,
        0.0,
        1.0,
        0.0,
        "1", // <- new parameter, represents the version of free to use. can be "1" or "2"
        ...
      ]
    }
  }

Note that this might break your code again the next time the extension is updated, which is why I strongly advise to use the dict API instead.

[kft334]

I didn't think that anything changed in v2 regarding the arguments/request format. Kinda figured that the issue was on my end. Thanks.