[feature request] ability to attach example workflow to node

[mholub]

Hello

I've recently started using ComfyUI and I am impressed by it

Long time ago I've been user of another node-based software called vvvv: https://visualprogramming.net/ One thing I really liked there was that each node could have an associated workflow with it which usually explained all parameters and did show how to use it properly.

https://youtu.be/7m1EzfxUtzo?list=PL2KeRstDQVRRVnzCHEambwAI4yWmpIF-p&t=324 - you can see example of it there

It would be cool to support it in ComfyUI too. Right now the blocker for it I believe is lack of tabs support or something like that - so some way to switch between workflows

But as first implementation I wouldn't really mind if this will be destructive i.e. you add node, press F1 and get a dialog: "Clear current workflow and open example for node KSampler instead?"

This will encourage custom node creators to document their nodes more and in the end this will boost the whole ecosystem of ComfyUI a lot

It feels like this feature should be somewhat easy to implement since ComfyUI already supports of saving the whole workflow into json or image file

[mholub]

Oh, and btw, @comfyanonymous, if you haven't seen vvvv software - I really recommend to spend a weekend and get familiar with it

It is quite old and it solved lots of UX issues in very smart ways, you might get some inspirations from it

[MichaelMcCulloch]

For clarity, do you want example usage attached to just the base nodes, or custom nodes too?

[mholub]

It makes a lot more sense for custom nodes since base nodes you learn pretty fast anyway

I don't see a reason why would I do it differently for base or custom unless it's some technical limitation?

Not sure what would be the best way to define it too I am not familiar with comfyui source code From what I see nodes are classes with some metadata i.e. https://github.com/biegert/ComfyUI-CLIPSeg/blob/main/custom_nodes/clipseg.py

It could be something like EXAMPLE="examples/clipseg.json" inside I guess?

[NeedsMoar]

vvvv appears to just be a variant of Windows Workflow Foundation which has shipped with visual studio since the early .NET days with some extra layout complication added and a far worse design; WWF was integrated into visual studio and used intellisense for everything, custom code could be inserted into any node in any .NET language, etc.

Still, I don't think having a question mark at the top of each that led to at least the github page of the node in question would be a bad idea (although lots of the custom node authors couldn't be bothered to document things there either). Forcing custom nodes to not have the same names as base nodes and to declare some minimal namespace (author name usually) and node type (i.e. sampling, utils, etc) would go a long way towards making installing a lot of them more reasonable. I currently have maybe 1/3rd of the stuff from the manager installed and the "add node" menu takes up 2/3 of a 2560x2880 display vertically with a ~10 point font in the UI... and I'm using the only display on the market with that much vertical space, most people are dealing with 16:9 so having menus that went further out horizontally than vertically would help them even more.

At the same time finding the couple of nodes I might actually wanted to use out of the pack of 40 (most of which are duplicates of something else or just integer / text inputs) is practically impossible because I tend not to remember that I should be looking for a single node SDXL + refiner under a menu with somebody's handle as the name. Enforcing organization and unique (and descriptive) names would go a long way towards not needing this feature for most custom nodes, I think. Maybe require mixins or decorators on the main node classes? I mean, this is python so it's not like the language is going to be able to figure out if something is missing until it tries to find it, but at least those are a start.

Diffusers makes good use of mixins in their pipelines to tack LoRA / Controlnet / etc loading onto base pipelines. The remaining few that are complicated enough tend to be samplers and conditioning nodes that behave in different ways... but some of those don't describe what all their options do on their githubs, either. None of the conditioning nodes I've tried explain what the difference between comfy and comfy++ and compel are or what the weighting options do, or whether they support directional weighting with < and > like the DirectionalConditioning node which is broken thanks to API changes which was all I've been trying to find without reading miles of python source. One of the KSamplers I tried uses some strange process of denoising to a point, then trying to reverse and predict the noise, then back again, but what good values for any of it might be aren't documented.

Of the root UI I'd say the only things that might be documented better are the samplers. There are a lot of them now and not much clear indication of when to use which. I do like the dpm++3m_sde node that was added more recently since it's able to produce very good results in few iterations and that's really important when you're dealing with crap-lousy DirectML and trying to run on an upscaled latent. Still I don't know where it shouldn't really be used for whatever reason. UniPC is odd with some models and gives cubist style output but normal looking with others. I studied mathematics up to graduate level before going to do something else so I'm a bit rusty but can understand the math itself, it just doesn't have much meaning without more knowledge than I'm willing to dig up on how the networks are structured; a simple suggestion on those about what type of output a given sampler might work best with would be enough to get anybody started.

It sort of reminds me of the "TTA" option on all of the command line upscalers (realsr / waifu2x / realesrgan / realcugan) as well as the frame interpolators. They all have it. Zero of them say a single word about what it does, but it slows things down a lot and there's presumably some reason for it to exist. Whatever it changes is subtle enough (like many of the sampler variants again here) that I couldn't tell the difference testing it on video

[aegis72]

I feel like Comfy is where Blender and other OSS was at one point :). Some rigor to maintain decent UX of nodes would be really cool and most welcome, although tough to enforce. I truly appreciate those devs that document well.

In general the AI space has a LOT of inside baseball, with names chosen for maximum-ingroup and minimum memorability. They might work for academics, but for usability, not so much.

Even a text note attached to nodes (or customizable by the user) would be welcome. "Here's a URL to get more info. This node does X, Y, and Z and is often used to 1, 2, 3."

TLDR, your post is great and you're great for making it :).