reddigari
commented
4 years ago @reelmatt react-diagrams allows different port subclasses, each implementing their own canLinkToPort() method --- this will let us validate connections. We can also parameterize the number of flow ports (e.g. num_flow_in), and render them on the visual node accordingly.
Awesome work here! Can't believe you got this going this so quickly.
How FlowNodes work as flow variables
Global flow variables are associated with a workflow (in KNIME, right-click on workflow -> 'Workflow Variables...') and local flow variables behave like regular nodes in the graph (with special red circle ports indicating a flow variable).
In KNIME, the two key pieces of information a flow variable provides are the 'var_name' and the 'default_value'. In the context of
pyworkflow, the 'var_name' would substitute a Node option (e.g. 'filepath_or_buffer' forReadCsvNode) and the 'default_value' would replace the value associated with that option.FlowNodes are currently similar to any other Node, and local FlowNodes are created the same way. To create a global FlowNode,
is_global: trueis passed in the POST body when creating the node. Read/update/delete are performed the same way as other nodes, just at the/node/global/<node_id>endpoint. Global FlowNodes are stored in a new NetworkX Graph that is added to the Workflow and saved with the JSON output.Existing endpoints should behave the same way, but some function signatures were changed slightly:
node.py:execute()now takes aflow_varsparameterworkflow.py:retrieve_node_data()is still a static method, but accepts a Node object as the argument instead of(workflow, node_id). This moves Node lookup out of the retrieval step and prevents cases where the same Node is retrieved twice.workflow.py:to_graph_json()is now a static method that accepts a NetworkX graph, to work for both the 'graph' (workflow) and 'flow_vars'.Limitations/TODO
This initial implementation should serve as a good starting point for further development. Currently, if a FlowNode is connected to a Node's in-port, the FlowNode variable will replace that option within the Node. Instead, this variable should be provided as an option the user can select when configuring the Node (i.e. use this manual value OR the value from the local flow variable).
Likewise, one can perform all CRUD operations on global FlowNodes, but they cannot be connected to actual Nodes. The
/workflow/globalsendpoint returns the list of currently defined global flow variables which should be provided as a drop-down for appropriate fields when configuring a Node.This gets at @reddigari's point from #41 re: passing Node options into pandas calls as
kwargs. We will probably want to investigate another approach.There's also the question of different 'types' of ports.
ReadCsvNodehas 0 in-ports, but should accept 1 in-port for flow variables. ThePivotNodehas 1 in-port, but should have another in-port for flow variables. Is this a front-end approach where react-diagrams can prevent non-like ports from connecting? Does the back-end need to differentiate between port types or is knowing the type of Node connected enough?