Goals / Ideas

[melMass]

Hey Patricio,

I am making an issue to better talk about how I could help. I maintain the mtb nodes: https://github.com/melMass/comfy_mtb/ which leverages a few interesting tricks that I think could apply here. (like dynamic inputs)

I just published my old POC here for reference: https://github.com/melMass/ComfyUI-Lygia, I basically did not got much further because I wanted to think properly about the architecture.

I know comfy quite well by now so with your expertise on lygia (and node systems) and mine on comfy's front/back I think we should be able to tackle things quickly.

[patriciogonzalezvivo]

Hi @melMass ! Thank you for hopping here! Getting dynamic inputs for textures and uniform would be amazing!

My plan is:

• have * inputs.
• if you plug a single image becomes a single textures uniform sample2D u_tex<NUMBER>, if it's a list of images it's an array of textures under uniform sample2D u_tex<NUMBER>[]; that way a list of images that makes a short video can be parse through u_frame. I think we inject a #define U_TEX<NUMBER>_TOTALFRAMES with the length of the array.
• if you connect a single float value becomes a uniform float u_val<NUMBER>, if it's a list of floats under 4 elements becomes a vec2, vec3 or vec4. if it's 9 elements mat3, if it's 16 elements a mat4.

How that sounds? it's possible? I'm going to check your repo and see if I can get started with that

The next step would be to add a JS IDE for code interface on the node. So users can have a better experience. I saw this node that already figurate out part of it https://github.com/sofakid/dandy

Any help on either of both task will be super welcome.

Thank you again!

[melMass]

• * inputs are possible! that's what I use in dynamic inputs. The core of the "hack" is to declare the node method with **kwargs, and handle the proper type casting on connect on the js side.
• That should be the "casting" part I just mentioned, it will end up as a mix of js and py for the dynamism.
• This is one of the current issues IMO in comfy. These should all be builtin types, currently, you'd have to make custom types (which we all end up doing, but I think it's a mess since they might not end up being compatible between node packs)

For text editing with highlights I use ACE in note+ it's relatively lightweight:

[patriciogonzalezvivo]

Thank you for the fast response! Went a head and added support for Textures / TexturesArrays I have to test it first. But I belive this will make easier any further improvement making it dynamic.

About the code editor. That seems nice. I will keep looking to https://github.com/sofakid/dandy. Having a mature IDE with hotkeys and multi selection, etc. would be a nicer experienced. I think.

I will test the TextureArray and then look into the IDE.

Any help on the dynamic inputs will be super appreciate it.

[Amorano]

Hi @melMass ! Thank you for hopping here! Getting dynamic inputs for textures and uniform would be amazing!

My plan is:

* have `*` inputs.

* if you plug a single image becomes a single textures `uniform sample2D u_tex<NUMBER>`, if it's a list of images it's an array of textures under `uniform sample2D u_tex<NUMBER>[];` that way a list of images that makes a short video can be parse through `u_frame`. I think we inject a `#define U_TEX<NUMBER>_TOTALFRAMES` with the length of the array.

I already have dynamic types being parsed at use time -- you can just rip the code from there.

web/nodes/glsl.js

and

sup/shader.py

Dont forget the OGL node from FIL too =D

Welcome to the club!

[Amorano]

Oh yes, keep in mind, mine supports VEC* types and dynamic casting of vars to input types, so, it is a little more robust than just doing dynamic types at this point.

Just something to keep in mind since there are uniform ivec and vec types, etc...

[melMass]

Dont forget the OGL node from FIL too =D

Can you link it? You also mentioned it on twitter but I can't find it

[Amorano]

Dont forget the OGL node from FIL too =D

Can you link it? You also mentioned it on twitter but I can't find it

https://github.com/filliptm/ComfyUI_Fill-Nodes

based on 100% shadertoy compat -- think it has audio reactive support/audio input support