search

DrCyanide / cyanic-sd-krita

A Stable Diffusion plugin for Krita, compatible with A1111 and SD.Next
68 stars 4 forks source link

A1111 ControlNet API mismatch #39

Closed Rinkaa closed 3 months ago

Rinkaa commented 3 months ago

Thank you for your great plugin for Krita!

I noticed a minor issue. When hitting 'Generate' button with ControlNet enabled, an error is thrown in A1111 sd-webui terminal:

Environment: cyanic-sd-krita: master sd-webui: 1.9.3 sd-webui-controlnet: a5457dcc (May 19)

*** API error: POST: http://127.0.0.1:7860/sdapi/v1/txt2img {'error': 'ValidationError', 'detail': '', 'body': '', 'errors': "2 validation errors for ControlNetUnit\nresize_mode\n   (type=assertion_error)\ncontrol_mode\n  value is not a valid enumeration member; permitted: 'Balanced', 'My prompt is more important', 'ControlNet is more important' (type=type_error.enum; enum_values=[<ControlMode.BALANCED: 'Balanced'>, <ControlMode.PROMPT: 'My prompt is more important'>, <ControlMode.CONTROL: 'ControlNet is more important'>])"}
    Traceback (most recent call last):
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/streams/memory.py", line 98, in receive
        return self.receive_nowait()
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/streams/memory.py", line 93, in receive_nowait
        raise WouldBlock
    anyio.WouldBlock

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 78, in call_next
        message = await recv_stream.receive()
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/streams/memory.py", line 118, in receive
        raise EndOfStream
    anyio.EndOfStream

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/ikigai/AI/stable-diffusion-webui/modules/api/api.py", line 186, in exception_handling
        return await call_next(request)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next
        raise app_exc
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro
        await self.app(scope, receive_or_disconnect, send_no_error)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 108, in __call__
        response = await self.dispatch_func(request, call_next)
      File "/home/ikigai/AI/stable-diffusion-webui/modules/api/api.py", line 150, in log_and_time
        res: Response = await call_next(req)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next
        raise app_exc
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro
        await self.app(scope, receive_or_disconnect, send_no_error)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 26, in __call__
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
        raise exc
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
        await self.app(scope, receive, sender)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
        raise e
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
        await route.handle(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
        response = await func(request)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app
        raw_response = await run_endpoint_function(
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/routing.py", line 165, in run_endpoint_function
        return await run_in_threadpool(dependant.call, **values)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
        return await anyio.to_thread.run_sync(func, *args)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
        return await get_asynclib().run_sync_in_worker_thread(
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
        return await future
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
        result = context.run(func, *args)
      File "/home/ikigai/AI/stable-diffusion-webui/modules/api/api.py", line 478, in text2imgapi
        processed = process_images(p)
      File "/home/ikigai/AI/stable-diffusion-webui/modules/processing.py", line 845, in process_images
        res = process_images_inner(p)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 56, in processing_process_images_hijack
        cn_is_batch, batches, output_dir, input_file_names = get_cn_batches(p)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 196, in get_cn_batches
        units = external_code.get_all_units_in_processing(p)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 133, in get_all_units_in_processing
        return get_all_units(p.scripts, p.script_args)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 146, in get_all_units
        return get_all_units_from(script_args[cn_script.args_from : cn_script.args_to])
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 174, in get_all_units_from
        all_units = [
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 175, in <listcomp>
        to_processing_unit(script_arg)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 228, in to_processing_unit
        return ControlNetUnit.from_dict(unit)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/args.py", line 420, in from_dict
        return ControlNetUnit(**values)
      File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
    pydantic.error_wrappers.ValidationError: 2 validation errors for ControlNetUnit
    resize_mode
       (type=assertion_error)
    control_mode
      value is not a valid enumeration member; permitted: 'Balanced', 'My prompt is more important', 'ControlNet is more important' (type=type_error.enum; enum_values=[<ControlMode.BALANCED: 'Balanced'>, <ControlMode.PROMPT: 'My prompt is more important'>, <ControlMode.CONTROL: 'ControlNet is more important'>])

---

I roughly remember a month ago when sd-webui was not 1.9.x yet, the generation had been working fine. So I guess it might be some recent changes from sd-webui or sd-webui-controlnet breaking this

DrCyanide commented 3 months ago

I haven't updated my local copy of Stable Diffusion in a long time, so it's likely there's an update that broke it. I suspect it's an update to the ControlNet extension specifically that broke it, given the error at the end is something that's specific to ControlNet.

I'll have to pull an updated copy and see if there's a way to tell which values ControlNet expects.

Rinkaa commented 3 months ago

I took a look at the plugin code, and saw it seemed some enums were moved here:

https://github.com/Mikubill/sd-webui-controlnet/blame/a5457dcc1c952a400b84a2ab90eade47c0d7fed7/scripts/enums.py#L244-L251C45

https://github.com/Mikubill/sd-webui-controlnet/blame/a5457dcc1c952a400b84a2ab90eade47c0d7fed7/scripts/enums.py#L259-L275C35

Hope these info help

DrCyanide commented 3 months ago

I just pushed a fix for this, so try pulling it and see if that works for you as well. I updated my local version of ControlNet to the latest to test it, so I don't have a convenient way to test backwards compatibility, but the fixes I made shouldn't break older versions.

The breaking change was documented here: https://github.com/Mikubill/sd-webui-controlnet/pull/2847

As of now ControlNet hasn't updated it's examples or documentation to reflect this breaking change. I've logged a ticket for those issues here: https://github.com/Mikubill/sd-webui-controlnet/issues/2920

Rinkaa commented 3 months ago

Thanks! I manually tested a few versions after pulling the new commit of cyanic-sd-krita, here is the test log with the versions I used and the results I observed. I used "My prompt is more important" and "Just resize" for each generation.

cyanic-sd-krita 1a4618f7 (Apr 15)
sd-webui v1.9.0 adadb4e3 (Apr 13)
sd-webui-controlnet v1.1.445 59d59988 (Apr 20)
ok
---

cyanic-sd-krita 64606ff1 (May 20)
sd-webui v1.9.0 adadb4e3 (Apr 13)
sd-webui-controlnet v1.1.445 59d59988 (Apr 20)
ok
---

cyanic-sd-krita 64606ff1 (May 20)
sd-webui v1.9.3 1c0a0c4c (Apr 13)
sd-webui-controlnet v1.1.447 3d75bd08 (May 4)  <-- Before the breaking change
ok
---

cyanic-sd-krita 64606ff1 (May 20)
sd-webui v1.9.3 1c0a0c4c (Apr 13)
sd-webui-controlnet v1.1.448 d037c2d3 (May 6)  <-- Right after the breaking change

*** API error: POST: http://127.0.0.1:7860/sdapi/v1/txt2img {'error': 'ValidationError', 'detail': '', 'body': '', 'errors': '1 validation error for ControlNetUnit\nlow_vram\n  value could not be parsed to a boolean (type=type_error.bool)'}
    Traceback (most recent call last):
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/streams/memory.py", line 98, in receive
        return self.receive_nowait()
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/streams/memory.py", line 93, in receive_nowait
        raise WouldBlock
    anyio.WouldBlock

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 78, in call_next
        message = await recv_stream.receive()
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/streams/memory.py", line 118, in receive
        raise EndOfStream
    anyio.EndOfStream

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/ikigai/AI/stable-diffusion-webui/modules/api/api.py", line 186, in exception_handling
        return await call_next(request)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next
        raise app_exc
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro
        await self.app(scope, receive_or_disconnect, send_no_error)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 108, in __call__
        response = await self.dispatch_func(request, call_next)
      File "/home/ikigai/AI/stable-diffusion-webui/modules/api/api.py", line 150, in log_and_time
        res: Response = await call_next(req)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 84, in call_next
        raise app_exc
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/base.py", line 70, in coro
        await self.app(scope, receive_or_disconnect, send_no_error)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/cors.py", line 84, in __call__
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/gzip.py", line 26, in __call__
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
        raise exc
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
        await self.app(scope, receive, sender)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
        raise e
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
        await route.handle(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
        await self.app(scope, receive, send)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
        response = await func(request)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/routing.py", line 237, in app
        raw_response = await run_endpoint_function(
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/fastapi/routing.py", line 165, in run_endpoint_function
        return await run_in_threadpool(dependant.call, **values)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/starlette/concurrency.py", line 41, in run_in_threadpool
        return await anyio.to_thread.run_sync(func, *args)
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 33, in run_sync
        return await get_asynclib().run_sync_in_worker_thread(
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 877, in run_sync_in_worker_thread
        return await future
      File "/home/ikigai/AI/stable-diffusion-webui/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 807, in run
        result = context.run(func, *args)
      File "/home/ikigai/AI/stable-diffusion-webui/modules/api/api.py", line 478, in text2imgapi
        processed = process_images(p)
      File "/home/ikigai/AI/stable-diffusion-webui/modules/processing.py", line 845, in process_images
        res = process_images_inner(p)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 56, in processing_process_images_hijack
        cn_is_batch, batches, output_dir, input_file_names = get_cn_batches(p)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/batch_hijack.py", line 196, in get_cn_batches
        units = external_code.get_all_units_in_processing(p)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 133, in get_all_units_in_processing
        return get_all_units(p.scripts, p.script_args)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 146, in get_all_units
        return get_all_units_from(script_args[cn_script.args_from : cn_script.args_to])
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 174, in get_all_units_from
        all_units = [
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 175, in <listcomp>
        to_processing_unit(script_arg)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/external_code.py", line 228, in to_processing_unit
        return ControlNetUnit.from_dict(unit)
      File "/home/ikigai/AI/stable-diffusion-webui/extensions/sd-webui-controlnet/internal_controlnet/args.py", line 390, in from_dict
        return ControlNetUnit(**values)
      File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
    pydantic.error_wrappers.ValidationError: 1 validation error for ControlNetUnit
    low_vram
      value could not be parsed to a boolean (type=type_error.bool)
---

cyanic-sd-krita 64606ff1 (May 20)
sd-webui v1.9.3 1c0a0c4c (Apr 13)
sd-webui-controlnet v1.1.449 cbc7ef33 (May 19)  <-- Latest labelled version
ok
----

cyanic-sd-krita 64606ff1 (May 20)
sd-webui v1.9.3 1c0a0c4c (Apr 13)
sd-webui-controlnet 7b4cdea2 (May 20)  <-- Latest commit
ok
----

The specific version that yelled an error is just a bug on their side as stated in https://github.com/Mikubill/sd-webui-controlnet/issues/2858, and I suppose it is fixed in https://github.com/Mikubill/sd-webui-controlnet/pull/2860 .

So the issue is solved! I can tell the backwards compatibility is preserved too. Thank you again for taking time investing this!