implementing groups

[zeffii]

For my own sanity i felt it was necessary to have a closer look at the mindfuck that is CustomGroup Nodes :) . I was especially interested in reusing as much of the PyNode api as possible, but I think @ly29 already concluded that also this requires us to pretty much roll our own.

I started a branch https://github.com/nortikin/sverchok/tree/group_experiment which hopes to get away from the use of the Frame and instead be more like the ShaderNodeTree with a back button and an overlay to show the parent NodeTree.

---

This is merely an exploration i'm not married to any of this code

here a list of things that need to be taken care of (feel free to interject / edit) in no order of importance.

• [x] socket rename
• [x] deleting existing sockets
• [x] moving existing sockets
• [x] socket type recast
• [x] handle reroutes
• [x] generate group from selected
• [x] generate group from selected + auto connect head and tail to monad IO, and parent node appropriately
• [x] show relevant properties in the sockets of the node tree
• [ ] sort relinks by Y location of periphery (aesthetic - avoid crossing noodles)
• [x] handle node group copy
• [x] handle node group rename
• [x] expand monad to ordinary nodes
• [x] handle IO/json
• [x] handle script nodes
• [x] possible space_data.node_tree (the [ nodetree icon ][ + New ][pin] ) should show the parent node tree name as in ShaderNodeTree ..
• [x] if possible keep old node BETA groups working.

---

Design Reference

lukas_t made his own node group for object_nodes: https://developer.blender.org/diffusion/B/browse/object_nodes/release/scripts/nodes/group_nodes.py

[zeffii]

I think maybe the monad inputs could be arrange by node.location.y so as to avoid crossing noodles?

[ly29]

A very sane thought, I can do some testing tomorrow.

Some sanity testing might be in order.

[zeffii]

SvMonadCreateFromSelected(Operator) would be good idea to offer a mode which does not auto connect peripherals, simple bypass for get_relinks() and relink().

[zeffii]

a nitpick .. not sure what the process is for removing menu items from NODE_MT_node and inserting our own implementation in its place. Is it even possible? ..I really don't like the current situation where there are items in that menu that don't apply to Sverchok...

[zeffii]

maybe Ctrl+G to group and Ctrl+Shift+G to group without peripheral relinks.

yay / nay ?

[ly29]

yay from me, how do does overriding and operator for certain node type look like btw? git did something messed up (or I did with git actually). will try to look into it...

[zeffii]

I remember a long time ago i / we asked LukasT about overriding the Add New operator for nodeview, so as to attach a fake_user to each new tree, but we never implemented it. Suggest to me that we either didn't understand it or the method was not so pretty..

re git: last time you did a git push on this branch I hadn't pulled before my latest commit,. In order to save time I deleted the local sverchok and cloned the repo again to get your changes.

[zeffii]

overloading the operator: https://github.com/nortikin/sverchok/issues/356#issuecomment-50598373

[ly29]

Yeah I messed up git, really sorry about that.
node/basic/group_exp.py Is broken. Should you or I fix it?

[zeffii]

.. i will revert to 185e5ff and push (should have been b6f443e)

[zeffii]

i split the code into a few files

ui/nodeview_keymaps.py
ui/sv_monad_panel.py
utils/sv_monad_tools.py

[ly29]

I give up, I will stop messing around with git.

[zeffii]

uchh i don't think i restored to the last commit..

[ly29]

I think I messed it up again, will remove my local copy and reclone when it is restored

[zeffii]

give me about 10 minutes..

[zeffii]

should be ok now. git pull --rebase if you need to. I should have restored to : b6f443e

[ly29]

Okay, now are talking.

• Monad in monad, I think this should be allowed but how editing works out is worth thinking about
• if you make a monad with one node, propose_io_location fails.

    min_x, max_x = min(*x_locs), max(*x_locs)

Could be changed to

    min_x, max_x = min(x_locs), max(x_locs)

The expansion of arguments isn't needed anyway.

Otherwise looking good.

[ly29]

Will make branch to experiment a bit with creating a dynamic class representing each node group.

[zeffii]

oh right -- will drop the expansion.. done 4ad5215

Monad in Monad could definitely be cool, and may be less complicated to implement than it seems.

[ly29]

It should just work, some assumptions might have to be dropped.

min and max are clever:

    x_locs = [n.location[0] for n in nodes]
    y_locs = [n.location[1] for n in nodes]
    min_x, max_x = min(*x_locs), max(*x_locs)
    min_y, max_y = min(*y_locs), max(*y_locs)

Could be

    locs = [n.location[:] for n in nodes]
    min_x, min_y = min(locs)
    max_x, max_y = max(locs)

[zeffii]

nice. i like that shorter approach

[zeffii]

(but it produces different results..)

[zeffii]

    x_locs = [n.location[0] for n in nodes]
    y_locs = [n.location[1] for n in nodes]
    min_x, max_x = min(x_locs), max(x_locs)
    min_y, max_y = min(y_locs), max(y_locs)

vs

    locs = [n.location[:] for n in nodes]
    min_x, min_y = min(locs)
    max_x, max_y = max(locs)

[zeffii]

because:

>>> max([[20,30],[40,50],[70,80]])
[70, 80]

[ly29]

Okay, I have to be more careful. I was sure there was clever way to do like that.

Anyway, onwards to creating dynamic node groups with properties. First commits up but not functional yet. See dynamic_monad_node

Doing this some assumptions change so everything that uses parent_node is also being reworked.

[ly29]

Some progress Before making group After making group and unlinking.

Still many issues.

[zeffii]

that's what we want :) code-wise I must confess I don't understand much of it yet ( yet, of course, none of the syntax is new and i am comfortable will dynamic class creation). I will put time in today to get working knowledge of what you're doing.

[ly29]

I going through a bit of sorting out dependencies hell, the problem with F8 development...

[zeffii]

https://github.com/nortikin/sverchok/compare/b6f443e...1bea57f nice.

Is some dependecy hell is my fault? ..because i'm not registering the classes inside sv_monad_tools, but rather importing them inside group_exp.py then registering, and there's a few more ..

[zeffii]

or (what i expect..) the registering / unregistering of dynamic classes... ?

[ly29]

Yes, kind of, where things belong needs to be looked over a little bit. But also I am slithy rusty with the subtler point of python.

[ly29]

The the dynamic class thing will come with it's own issues, but that is further down the road.

What works now:

• Creating a node group with relinks and getting sockets with properties
• processing of the node group

What does not work

• Further editing
• Or anything else basically.

What needs to be done:

• Updating a node group should trigger make_class_from_monad and update all instance of the class collecting links etc.
• The needs to be a sanity check to handle more than one property with the same name, done
• Update all the the tools/operators to use this interface

[zeffii]

The needs to be a sanity check to handle more than one property with the same name first come first serve, if two 'x' then 'x', 'x2'. if user renames manually to an identical name, then the other socket should receive the postfix (and this to cascade to the parent node).

seems like you are moving towards Lukastoenne's reference implementation, i'm OK with that.

[zeffii]

let me get this straight: each instance of a monad is a unique class registered whenever we ctrl++G. ?

also... i dont understand the appending to the socket here of data

+    in_socket = []
+
+    for socket in monad_inputs.outputs:
+        if socket.is_linked:
+            other = get_other_socket(socket)
+            prop_name = getattr(other, "prop_name")
+            if prop_name:
+                cls_dict[prop_name] = getattr(other.node.rna_type, prop_name)
+            data = [socket.name, socket.bl_idname, prop_name if prop_name else None]
+            in_socket.append(data)
+
+    out_socket = []
+    for socket in monad_outputs.inputs:
+        if socket.is_linked:
+            data = [socket.name, socket.bl_idname]
+            out_socket.append(data)

[ly29]

For each monad there is unique monad node class with the corresponding properties to show in the sockets. This monad node can have many instances.

The data (bad name) is collected into input_template and output_template, it is the for each socket data needed to create the socket.

# without properties
>>> D.node_groups['NodeTree'].nodes['Group Exp'].input_template
[['vectors', 'VerticesSocket', None], ['Angle', 'StringsSocket', None], ['vertices', 'VerticesSocket', None], ['Rotation', 'VerticesSocket', None]]

# with properties
>>> D.node_groups['NodeTree'].nodes['Group Exp.002'].input_template
[['y', 'StringsSocket', 'y']]

    def sv_init(self, context):
        self.use_custom_color = True
        self.color = MONAD_COLOR

        for socket_name, socket_bl_idname, prop_name in self.input_template:
            s = self.inputs.new(socket_bl_idname, socket_name)
            if prop_name:
                s.prop_name = prop_name
        for socket_name, socket_bl_idname in self.output_template:
            self.outputs.new(socket_bl_idname, socket_name)

[zeffii]

ok, monad_inputs are the input socket on (what i was calling) 'parent_node' ?

[zeffii]

or an interim structure (*put_template) used to populate the input sockets on the parent_node

[ly29]

monad_inputs are taken from the SvGroupInputsNodeExp in the monad.

There are also some changes in the order things happen in the monad creation.

[ly29]

I just made the changes needed to make it work a little bit. For example every edit operation, renaming etc there needs to be some mechanism to update all

[ly29]

Node instances of a monad.

The next step is to make clever update mechanism for supporting editing. I will meditate on this for a while I think.

[zeffii]

looking good!

a side node: hey maybe we can make the redux run inside a monad (experimentally) eventually. could help bootstrap / transition.

[ly29]

Actually for most cases of editing, changing the templates and updating sockets could be done without registering a new class. Just changing the input_template etc and update as needed.

[ly29]

Looking at the operator code it looks reasonably simple to extend to the fact that there might be more than one node to edit.

Aslong as each edit is atomic and only affects one thing at a time it might be easier to something like below.

    def execute(self, context):
        node, kind, socket = get_data(self, context)
        IO_node_sockets = getattr(node, kind)
        pos = self.pos

        if self.direction == 0:
            IO_node_sockets.remove(socket)     # I/O interface (subgroup)
            # parent_sockets.remove(parent_sockets[pos])
            for node in node.id_data.node_instances:
                  sockets = get_sockets(node, kind)
                  sockets.remove(sockets[pos])
        else:
            def wrap_around(current_idx, direction, member_count):
                return (current_idx + direction) % member_count

            IO_new_pos = wrap_around(pos, self.direction, len(IO_node_sockets)-1)
            IO_node_sockets.move(pos, IO_new_pos)

            #parent_new_pos = wrap_around(pos, self.direction, len(parent_sockets))
           # parent_sockets.move(pos, parent_new_pos)
           for node in node.id_data.node_instances:
                sockets = get_sockets(node, kind)
                new_pos = wrap_around(pos, self.direction, len(sockets))
                sockets.move(pos, new_pos)
      # then make sure to update monad to add new things
      make_class_from_monad(node.id_data.name)

[zeffii]

how to make a clear distinction between

• monads instances (edit one edit all )
  • or lock instances and only let edits take place in the 'master' monad.
• and unique duplicates (no edit propagation

[ly29]

I think the sane approach is to consider them like resuable blocks of code. So if you edit one you edit all instances of it, if you want to duplicate you need to explicitly create a copy. Click the edit button brings up the same monad.

Could even be that you have to expand the monad and create a new monad. A minimal interface to achieve this would suggest that approach.

[zeffii]

ok, so if you are in an instance we display a large button in the UI to 'Make Unique'.

[nortikin]

OMG.

[zeffii]

@ly29 perhaps start a new thread regarding the move towards dynamic nodes, and copy the todolist and untick those items which are not yet implemented/working in dynamic_monad_node,

this thread got too long already :) slow for browsing new stuff

[zeffii]

close for now.