Potential enhancements for points editor

[kevinramirez723]

I know it's still very early in development, and having tried it with your ComfyUI SAM 2 implementation, here are some thoughts I have that would help for that use case. I don't know where you plan to go with the node so pardon me if you have already considered these points and are already working towards something similar or have ruled them out for one reason or another.

• Adding a second set of coord_str/normalized_str so that both positive and negative points can be set on a single canvas. This would understandably increase complexity and require more keyboard shortcuts, but would help create cleaner workflows for end users.
• In a similar vein, ability to add multiple bboxes on a single canvas.
• Instead of a bg_image input that requires queuing or the redundancy of clicking and dragging/right clicking for an image already present in the workflow I feel it would be more natural for the node to be akin to a load image node with an image output and no inputs. Granted this might be an issue for video based workflows. I guess solutions to this could be to either make it support video (sounds time consuming) or make two separate point editor nodes for both use cases.

I'm also not opposed to trying my hand at implementing some of these thoughts but I'll wait until its less work-in-progress. Regardless, really appreciate the work you're doing!

[kijai]

I have thought of all that, what's limiting me is my lack of JavaScript skills, everything I've done I've had to learn first as I have no previous experience.

For the bg_image you can already use copy/paste with it so it's not that much trouble to just right click on an image, copy and paste it in. Alternatives I've thought of is possibly grabbing the image from the Loader node as it (or link to it) should be saved in the Loader node by same virtual link.

I kinda wanted the results of the segmentation to be shown in the editor too, but that would make endless loop with how ComfyUI execution is setup... not sure how to tackle that, at least without more JavaScript.

[kijai]

Negative points should now be in, also made the normalization a boolean rather as it's cleaner.

[grandparodeo]

Two more enhancement ideas:

Ability to scale the size of the points editor node without losing functionality.

• I have three running in a workflow and they take up a lot of space, which makes it prohibitive.
• I've tried rescaling the input image but that seems to muck up the points editor.

Ability to pull a specific frame from an input video.

• Right now I have two setup two video inputs. One to output the specific frame for masking and another for the actual output.
• There are workarounds currently (manually copy / paste an image), but I'd like to setup an automated workflow so I don't need to leave Comfy to get additional data to work with.

Brilliant work!

[jokero3answer]

[jokero3answer]

How to add more than three points, I left-click and right-click do not respond! @kijai

[kijai]

How to add more than three points, I left-click and right-click do not respond! @kijai

Shift+ left click for positive, shift+ right click for negative.