fofr / cog-comfyui

Run ComfyUI with an API
https://replicate.com/fofr/any-comfyui-workflow
MIT License
347 stars 77 forks source link

StyleAligned Workflow #106

Open JaviSt97 opened 2 weeks ago

JaviSt97 commented 2 weeks ago

Hi there, I've skimmed through the available nodes and I don't think the necessary ones are there. Essentially:

Each of them are required to make this work (Example workflow). Replicate does have a style aligned API from lucataco, but it's somewhat limited in that we can't really pick the checkpoint nor add any pre-conditioning such as IP Adapters.

Thank you !

fofr commented 2 weeks ago

Does Style Align offer anything better than Style Transfer from IPAdapters?

FizzNodes and ImpactPack custom nodes are already available.

JaviSt97 commented 1 week ago

Thanks for the reply ! Great good to know – as for Style Align I've found it to be incredibly more consistent in terms of actual style transfer , without any bleeding from the composition. I use it for pretty much anything consistency-related now.

On that note, Latent Vision published few days ago a new weight type for IPAdapter that supposedly singles out efficiently the unet block responsible for style transfer to avoid bleeding. Haven't tried it yet, I'll update this here to see if it's on par with style align performance asap.

JaviSt97 commented 1 week ago

For reference – This is the image used for style transfer:

I used the prompt illustration of a basket of cherry tomatoes, cut-out on white background for both. Here is the result using IP Adapter with precise style transfer

Here using Style Aligned :

While Style Align bleeds a bit of the reference image messiness, I find the resulting style much more aligned (whimsical, lighthearted ..) – The composition itself could then be improved with prompt weights and / or controlnets on top.

JaviSt97 commented 2 days ago

Any news ?