[Issue]: API issue background removal is always on

[sebaxakerhtc]

Issue Description

I try to access /sdapi/v1/cmd-flags but it doesn't work. It should request credentials, but returns Not authenticated instead

Version Platform Description

Windows, API

[vladmandic]

can you describe your exact setup (i'm assuming you're using --api-auth?) and workflow of the actual api requests?

[sebaxakerhtc]

can you describe your exact setup (i'm assuming you're using --api-auth?) and workflow of the actual api requests?

Thank you for fast response! Yes. I use --api-auth. Using AIYA bot. And it work for image generation when i cut a part of code (for auth check). But is better way to optimize code to use it with original and yours nice fork! Part of code:

    if global_var.gradio_auth is None:
        r = s.get(global_var.url + '/sdapi/v1/cmd-flags')
        response_data = r.json()
        if response_data['gradio_auth']:
            global_var.gradio_auth = True
        else:
            global_var.gradio_auth = False

needs to be edited like this (to work on yours fork):

    if global_var.gradio_auth is None:
        r = s.get(global_var.url + '/sdapi/v1/cmd-flags')
        response_data = r.json()
        if response_data['gradio_auth']:
            global_var.gradio_auth = True
        elif response_data['auth']:
            global_var.gradio_auth = True
        else:
            global_var.gradio_auth = False

But /sdapi/v1/cmd-flags not work properly

Also, if you can help with it - upscale not work too on your fork

15:46:40-289632 ERROR    API error: POST:
                         http://127.0.0.1:7860/sdapi/v1/extra-single-image
                         {'error': 'OSError', 'detail': '', 'body': '',
                         'errors': 'cannot write mode RGBA as JPEG'}
15:46:40-293637 ERROR    HTTP API: OSError
╭───────────────────── Traceback (most recent call last) ─────────────────────╮
│ C:\automatic\venv\lib\site-packages\PIL\JpegImagePlugin.py:640 in _save     │
│                                                                             │
│   639 │   try:                                                              │
│ ❱ 640 │   │   rawmode = RAWMODE[im.mode]                                    │
│   641 │   except KeyError as e:                                             │
╰─────────────────────────────────────────────────────────────────────────────╯
KeyError: 'RGBA'

The above exception was the direct cause of the following exception:

╭───────────────────── Traceback (most recent call last) ─────────────────────╮
│ C:\automatic\modules\middleware.py:69 in exception_handling                 │
│                                                                             │
│   68 │   │   try:                                                           │
│ ❱ 69 │   │   │   return await call_next(req)                                │
│   70 │   │   except CancelledError:                                         │
│                                                                             │
│ C:\automatic\venv\lib\site-packages\starlette\middleware\base.py:84 in      │
│ call_next                                                                   │
│                                                                             │
│                          ... 22 frames hidden ...                           │
│                                                                             │
│ C:\automatic\venv\lib\site-packages\PIL\Image.py:2432 in save               │
│                                                                             │
│   2431 │   │   try:                                                         │
│ ❱ 2432 │   │   │   save_handler(self, fp, filename)                         │
│   2433 │   │   except Exception:                                            │
│                                                                             │
│ C:\automatic\venv\lib\site-packages\PIL\JpegImagePlugin.py:643 in _save     │
│                                                                             │
│   642 │   │   msg = f"cannot write mode {im.mode} as JPEG"                  │
│ ❱ 643 │   │   raise OSError(msg) from e                                     │
│   644                                                                       │
╰─────────────────────────────────────────────────────────────────────────────╯
OSError: cannot write mode RGBA as JPEG

[vladmandic]

re: generate with auth yes, i've renamed internal variables, but you should not be checking for state of internal variables anyhow: (orignal) response_data['gradio_auth'] vs (here) response_data['auth'], you should be checking for http status code 401 which means unauthorized

re: cmd-flags you say It should request credentials, but returns Not authenticated instead - but that's the whole point, endpoint doesn't REQUEST auth, it returns 401 unauthenticated if its unauthenticated. does it work if you send authentication credentials?

re: upscale when you're sending image as b64 via api, server doesn't know what do you want with it, so it will by default try to save in a format specified in options -> image options -> file format but if you send RGBA image, and default is set to JPG here (original has PNG as default), it will fail. simply either convert image to RGB before sending or set output format to PNG. i'll improve error handling around that, but in general it works as designed.

btw, everything i wrote applies to both here and original a1111 - if design is correct, it should work on both without any special handling.

[sebaxakerhtc]

simply either convert image to RGB before sending or set output format to PNG

I'm using url of image as init image - it very easy. Instead i need to download image, change to png and then send it to program... Looong way :) Can you give me info on how to return the work of upscale from url?

btw, everything i wrote applies to both here and original a1111

On original a1111 all works perfect. But yes, you're right So, i edited code like this:

    if global_var.gradio_auth is None:
        r = s.get(global_var.url + '/sdapi/v1/cmd-flags')
        if r.status_code == 401:
            global_var.gradio_auth = True
        else:
            global_var.gradio_auth = False

[vladmandic]

perfect. i'll also add check of image format vs number of channels in global image handler (this is not api specific) so it prints warning and converts image to RGB if desired format does not support RGBA (instead of runtime error as it is right now).

[sebaxakerhtc]

Sorry for your time here, but there's 1 more problem. Changed in settings format to PNG - upscaler start working, BUT it removes background!

[vladmandic]

most operations in SD do not support RGBA, very few do.

[sebaxakerhtc]

I mean integrated BG remover work i think, but i don't need it

It's an extension BUG mentioned here and have a PR, but unclear why it's not merged... Please pay attention to this because it in the extensions-builtin. Thank you for your hard work!

[vladmandic]

ahhh, that's nasty. yes, that should be fixed upstream. i cannot modify it here since rembg is a git submodule. i'll try to think of something.

i'm going to reopen this issue and mark it upstream so i can track it.

[prog0111]

Am also having this issue. I've gotten the upscaler API to work by picking PNG format instead of jpeg in settings, but all upscaled images remove background. I don't quite follow most of the conversation here, though... I can help test any changes if needed.

[sebaxakerhtc]

Am also having this issue. I've gotten the upscaler API to work by picking PNG format instead of jpeg in settings, but all upscaled images remove background. I don't quite follow most of the conversation here, though... I can help test any changes if needed.

All you need to do is https://github.com/AUTOMATIC1111/stable-diffusion-webui-rembg/pull/13/commits/7970e27901515a7e33ce37a96baa1737b52980da Works perfect

[prog0111]

Am also having this issue. I've gotten the upscaler API to work by picking PNG format instead of jpeg in settings, but all upscaled images remove background. I don't quite follow most of the conversation here, though... I can help test any changes if needed.

All you need to do is AUTOMATIC1111/stable-diffusion-webui-rembg@7970e27 Works perfect

Can confirm that this worked for me.

I modified extensions-builtin\stable-diffusion-webui-rembg\scripts\postprocessing_rembg.py and changed the line shown in the link above. Backgrounds are no longer always removed when using the API.

[vladmandic]

fixed.