[BUG] - Doesn't work for ComfyUI SDXL flows

[kelzan]

Has worked great up until now for ComfyUI generated images, however there's something about the new SDXL flows where it won't read the metadata. I've tried several flows, both version 0.9 and 1.0, but haven't gotten anything to work. I attached an example image generated by ComfyUI with SDXL1.0. Love this tool, by the way!

[receyuki]

@kelzan @GlebIgorevich @Marscog @Tikishark777 @lilixil Which solution do you guys prefer?
❤️ for option 1 and 🚀 for option 2

1. Always display Clip G, Clip L and Refiner prompt in separate textboxes.
2. Use tabs to switch between different prompts.

   For me, the second solution looks better, but I'm not sure if it's necessary to keep Clip G, Clip L and Refiner prompt displayed all the time.

[Marscog]

❤ I like option 1. I like to see all the info at once. -thanks On Sun, Aug 20, 2023, 10:25 AM Rhys Yang @.***> wrote:

@kelzan https://github.com/kelzan @GlebIgorevich https://github.com/GlebIgorevich @Marscog https://github.com/Marscog @Tikishark777 https://github.com/Tikishark777 @lilixil https://github.com/lilixil Which solution do you guys prefer? ❤️ for option 1 and 🚀 for option 2

1. Always display Clip G, Clip L and Refiner prompt in separate textboxes.

[image: 截屏2023-08-20 下午9 08 58] https://user-images.githubusercontent.com/28808141/261857317-dc6f9eb1-692f-4d5a-a04e-a72fff46924e.png

2. Use tabs to switch between different prompts. [image: 截屏2023-08-20 下午10 10 25] https://user-images.githubusercontent.com/28808141/261857329-68de0e3c-dfa6-420b-9ca7-2651ed4e4f99.png For me, the second solution looks better, but I'm not sure if it's necessary to keep Clip G, Clip L and Refiner prompt displayed all the time.

— Reply to this email directly, view it on GitHub https://github.com/receyuki/stable-diffusion-prompt-reader/issues/25#issuecomment-1685298565, or unsubscribe https://github.com/notifications/unsubscribe-auth/BB3Y7RKMVXPTQWGQWB2L45LXWIM53ANCNFSM6AAAAAA2ZTDHMU . You are receiving this because you were mentioned.Message ID: @.***>

[Tikishark777]

I like the second option as well. It's less cluttered and since I mostly only use clip g it fits my purpose. However, I'd be happy with either version. Ty for your time on this project.

[kelzan]

If the prompts are short, the first option is nice, however I think I'd prefer the second option the majority of the time.

[GlebIgorevich]

I think we need both types and the button for switching between them. 1st is good for the short prompts, but is not good for the long prompts. 2nd is opposite: good for long prompts only.

But it will be broken again if the workflow has more than 3+3 nodes for prompt. And no ideas how to showing that correctly. Merging all of them is not good idea.

[receyuki]

I think we need both types and the button for switching between them. 1st is good for the short prompts, but is not good for the long prompts. 2nd is opposite: good for long prompts only.

But it will be broken again if the workflow has more than 3+3 nodes for prompt. And no ideas how to showing that correctly. Merging all of them is not good idea.

Although I know that theoretically ComfyUI can have more than 2 Ksamplers, I don't understand in what situations users would need more than 3+3 prompts. Do you have any examples for that?

[receyuki]

I finally decided to keep both and switch between them using a button.

[Tikishark777]

Looks great. Good choice.

[receyuki]

I just released a new beta version🚀. Please test it with different sdxl images. If there are any issues, please let me know.

https://github.com/receyuki/stable-diffusion-prompt-reader/releases/tag/v1.3.3-beta.1

[GlebIgorevich]

Tested on the different old images which can't read via tool. Almost all can be read now. But a few still not working.

[Tikishark777]

Unfortunately, it won't work with any of my images. I do see text for the last box, but the everything is blank.

[receyuki]

@GlebIgorevich

    "77": {
        "inputs": {
            "filename_prefix": "ComfyUI_upscaled"
        },
        "class_type": "SaveImage"
    }

This workflow looks so weird. Although this node looks fine in ComfyUI, it's not connected to any other nodes in the raw workflow data. It appears to be a bug in ComfyUI or UltimateSDUpscale. But anyway, I've added new logic to handle this situation. Thank you for your help!

[receyuki]

SD-Prompt-Reader-1.3.3-beta.1+dev.1-Windows-x64.zip

@GlebIgorevich Try this version, images similar to the one above should all work fine now. If there are other images that are not working, please let me know.

@Tikishark777 Try this version first. If it still doesn't work properly, please upload the original image here or send it to my email at receyuki@gmail.com.

[Tikishark777]

Thank you for looking at this further. I tried the updated version, but still with no luck. I'm attaching an image from my workflow. It could be one of my extensions that's throwing everything off.

[GlebIgorevich]

@receyuki still not working.

Found the second image which can't read.

[receyuki]

@Tikishark777 Oh my goodness, this workflow is truly impressive. I've never seen such a complex workflow before, and almost every node is a custom node.

However, this also means that I'll need a large amount of time to support these custom nodes. Especially with Searge, this extension has significantly different characteristics compared to the original ComfyUI.

Thus, I'm sorry to say that I might not be able to provide support for Searge. Nevertheless, you can still use the copy button next to the export button to extract all the raw metadata. Although you may not be able to directly discern any useful infomation from it.

[receyuki]

@GlebIgorevich I will be releasing beta2 within the next few days to fix the norwegian forest cat. But what I don't understand is that the mermaid image works fine on my end. I guess you might be using the wrong version. But don't worry, give beta2 a try when it's released, and we'll see how it goes.

[Tikishark777]

I completely understand. I do appreciate all you hard work though. I'm currently trying to replicate some of what Searge is doing without all the custom nodes. Hopefully, that will help as well.

[receyuki]

@Tikishark777 In theory, as long as the workflow uses native KSampler nodes, including "KSampler", "KSamplerAdvanced" and native CLIPTextEncode nodes such as "CLIPTextEncode", "CLIPTextEncodeSDXL" and "CLIPTextEncodeSDXLRefiner" the workflow should be recognised correctly. But, I am aware that this requirement is quite demanding.

[receyuki]

@GlebIgorevich https://github.com/receyuki/stable-diffusion-prompt-reader/releases/tag/v1.3.3-beta.2 I just released beta 2. Try it out.

[GlebIgorevich]

@receyuki works better! All the previous problems looks like fixed.

But I found an another problem: seed, image size and models can't displayed.

Try this image: https://civitai.com/images/2247572

[receyuki]

@GlebIgorevich Can you post the image here? I can't download the original image from civitai. When I download the images, they are all being compressed into JPEG.

[GlebIgorevich]

@receyuki done

[receyuki]

@GlebIgorevich This new bug is again caused by new custom nodes, and I will fix it in the next release.

However, to be honest, the more custom nodes users use, the higher the likelihood that image data won't be read correctly. Civitai's online tools tend to ignore custom nodes, while the diffusion toolkit only directly reads data from KSampler, so it's also unable to handle custom nodes. Perhaps I should consider giving up support for custom nodes.

[Marscog]

None of my comfy files work. I think it is because I use SDXL Promp Styler for everything. Here is an example file. I think it would be difficult to get it to work with comfy since there are infinite workflows.

[Marscog]

I'm using a new program now, Fooocus the Moonrider branch. I don't know if you could read the info from these files. I'm sure it would be easier than comfy. Here is an example.

[receyuki]

@Marscog Can you open a new issue of type "Format Support Request"? I took a quick look, and I believe I should be able to support this format quite easily.

BTW, I just fixed the bug of the Ben Grimm image. The issue was actually caused by "ControlNetApplyAdvanced." As for "SDXLPromptStyler", I added support for it a few versions ago.

[receyuki]

@GlebIgorevich @Marscog SD-Prompt-Reader-1.3.3-beta.2-dev.1-Windows-x64.zip Try it out. (Hopefully the last dev version)

[Marscog]

The new version works for my files in comfy and fooocus MRE branch. THANKS!

[GlebIgorevich]

@receyuki That's working. Thank you a lot!

Ideas for the next update: collecting info form Face Detailer node (this is a same as After Detailer for A1111) and upscaling (like a hires).

[receyuki]

@GlebIgorevich Both FaceDetailer and UltimateSDUpscale are custom nodes, and personally, I would prefer not to handle the parameters of custom nodes, as this would be another never-ending task.

Also, both FaceDetailer and UltimateSDUpscale have very large number of parameters, even more than KSampler. If I include the custom node parameters in the settings, the settings will become really really long.

[GlebIgorevich]

I think supporting the ComfyUI workflows is never-ending task anyway. Just because someone will add the new custom basic nodes (prompts, samplers, styles).

[receyuki]

I just released the SD Prompt Reader ComfyUI node version, try it out. If you encounter any issues or have any suggestions, please let me know. https://github.com/receyuki/comfyui-prompt-reader-node

[WinstonWoof]

wonderful concept however point blank refuses to read anything saved with a standard ComfyUI Image Save node for me (### ComfyUI Revision: 1866 [1b103e0c] | Released on '2023-12-30')

Example Image attached

NB using the Node version I get this

[receyuki]

@WinstonWoof The image above uses many custom nodes that are not supported by the SD Prompt Reader. It's best to use Prompt Saver node to ensure the metadata can be read by Prompt Reader or any other metadata reading tool.

The reason for this is quite simple: for each custom node, I have to support them individually. Although I have supported a large number of custom nodes, this is still an endless task. That's why I decided to make Prompt Saver node.