search

WASasquatch / was-node-suite-comfyui

An extensive node suite for ComfyUI with over 210 new nodes
MIT License
1.15k stars 170 forks source link

REQUEST: Blip node #26

Closed ghost closed 1 year ago

ghost commented 1 year ago

Can we get a blip node? Ideally this would take in a blip model loader, an image and output a string. This would allow us to combine a blip description of an image with another string node for what we want to change when batch loading images.

WASasquatch commented 1 year ago

Sounds like a good idea. Though I find BLIP terrible with Stabke Diffusion since it uses CLIP ViTL14 for it's text encoding. That's why I personally use CLIP Interrogator, it's more accurate to the tags that are used in SD 1.4/1.5

ghost commented 1 year ago

I tend to use sd2.x but would a choice of blip or clip (or I believe there are others) be possible? The more choice the better lol Not sure on the limitations of custom nodes and if it's even possible but it would certainly be useful on batch image processing

WASasquatch commented 1 year ago

I tend to use sd2.x but would a choice of blip or clip (or I believe there are others) be possible? The more choice the better lol Not sure on the limitations of custom nodes and if it's even possible but it would certainly be useful on batch image processing

As far as the nodes go, this sort of task is sorta perfect for them. It's the dynamic stuff I struggle with. Like taking input, and showing results on the node. We need this functionality really bad. Fingers crossed it's on high priority over at ComfyUI.

I'll take a look at what these entail. BLIP would probably be where to start as it is (I believe at least) a little more contained then CLIP Interrogator which is more just a large script.

ghost commented 1 year ago

much appreciated. Yeah having the nodes be able to receive and display dynamic text would be handy, I used a chatgpt node from another custom node (yes I confess to seeing other nodes lol) and the prompt it got could be displayed in the command window but it would make more sense to have it displayed in a node on the app. That functionality and the SD Upscale Script from automatic1111 making its way to comfyUI are two things I think need changes in the core code to work?

for now happy to see achievable stuff coming and very happy with the stuff you have added so far (much of it really should be core functionality like the upscale by ratio)

WASasquatch commented 1 year ago

much appreciated. Yeah having the nodes be able to receive and display dynamic text would be handy, I used a chatgpt node from another custom node (yes I confess to seeing other nodes lol) and the prompt it got could be displayed in the command window but it would make more sense to have it displayed in a node on the app. That functionality and the SD Upscale Script from automatic1111 making its way to comfyUI are two things I think need changes in the core code to work?

for now happy to see achievable stuff coming and very happy with the stuff you have added so far (much of it really should be core functionality like the upscale by ratio)

Don't you find SD Upscale really bad? Maybe I was using it wrong, but i felt it had really bad quality over just doing a HR-Fix at a huge size. Like with ComfyUI I can up to about a 4k image.

ghost commented 1 year ago

sd upscale script an image with the same prompt and a denoise of .25 to .35, tile overlay of 96 and something like 4x_foolhardy_Remacri as the upscaler and you get something of an amazing result. you can see it doing each section of the image. In comfyui though, start going bellow 0.5 denoise and your in for a glitchy image. I have yet to find a method to get results as crisp and clear when you zoom in as SD Upscale Script delivers.....but the key really is that low denoise too high and you get a mess :)

WASasquatch commented 1 year ago

Oh yeah, that's originally called "GO BIG". From ProgRockDiffusion. I added the script to Easy Diffusion ages ago when Stable Diffusion came out. Originally from Disco Diffusion. I've been thinking of adding similar, but I don't think comfyui has color fix, so the tiles would differ a lot.

ghost commented 1 year ago

Oh yeah, that's originally called "GO BIG". From ProgRockDiffusion. I added the script to Easy Diffusion ages ago when Stable Diffusion came out. Originally from Disco Diffusion. I've been thinking of adding similar, but I don't think comfyui has color fix, so the tiles would differ a lot.

I had assumed it was impossible to add to comfyui, or that would have been my first request a long time ago (think i have requested it on the scripts github lol) I have found acceptable upscaling on comfui but nothing that gets the same level of detail when zoomed in (at the same resolution) as sd upscale script can do. The rest of comfyui makes up for that short fall though, I love the node system. shrug I cant see me going back to auto just for the script though. when i want to do some deforum is the only time i load it up now

WASasquatch commented 1 year ago

Yeah I know what you mean, and ComfyUI should only get better.

FYI working on BLIP. Been a real chore. Spent all day figuring out I was trying to get a install working that actually didn't need to be installed and BLIP kinda put it as a dependency for no reason. That wasted so much time.

WASasquatch commented 1 year ago

BLIP is now added, hopefully it's good across hardware/operating systems.