Add a dedicated render method

[yellow1912]

I understand that this request has been posted and closed here: https://github.com/alyssaxuu/flowy/issues/40

However, I would like to bring it up and present some reasons why I think it will be very useful:

1. In many cases we will need to save the tree in a relational database, only to query and re-build it on demand. Having to store the whole tree in html is not efficient in term of storage.
2. What if we change our design/presentation at some point? It also makes it almost impossible (or very difficult) to change the tree structure via other methods (such as database query) because you will have to somehow re-render the html tree.
3. Right now it seems like the snapping callback is where we can update the presentation of the snapped blocks. I would argue that having a dedicated render method allow us to break free from the need to rely on snap event (and thus also makes adding blocks manually via API possible).

And while we are at it, I would also argue that we should NOT pass the attr together with the blocks (at least not to the render function, because render function only expects the block information and data which can be passed manually). I do understand why we have attr: right now we have the draggable blocks initialized as dom elements which is good but at the same time limited.

We should not mix dom objects with block objects. A block object should have:

1. Id
2. Parent id
3. Name
4. Additional meta data if any

{ "id": 1, "parent": 0, "data": {"name": "blockid", "value": "1" } }

For the html dom based draggable blocks, they can still be coded like this:

<div class="create-flowy" data-id="1" data-parent="0" data-data='{"name": "blockid", "value": "1"}'>Grab me</div>

or perhaps:

<div class="create-flowy" data-id="1" data-parent="0" data-name="blockid" data-value="value">Grab me</div>

One last thing is regarding the current api, perhaps if we switch to this it will be easier to allow easier change in the future:

flowy(canvas, ongrab, onrelease, onsnap, onrearrange, spacing_x, spacing_y);

to

flowy({canvas: element, onGrab: function, onRelease: function, onSnap: function, onRearrange: function, render: function, spacingX: number, spacingY: number});

or (only canvas is a must, all other things are optional)

flowy(canvas, {onGrab: function, onRelease: function, onSnap: function, onRearrange: function, render: function, spacingX: number, spacingY: number});

[alyssaxuu]

• In many cases we will need to save the tree in a relational database, only to query and re-build it on demand. Having to store the whole tree in html is not efficient in term of storage.
• What if we change our design/presentation at some point? It also makes it almost impossible (or very difficult) to change the tree structure via other methods (such as database query) because you will have to somehow re-render the html tree.
• Right now it seems like the snapping callback is where we can update the presentation of the snapped blocks. I would argue that having a dedicated render method allow us to break free from the need to rely on snap event (and thus also makes adding blocks manually via API possible).

I see why it would be useful - as I understand it you'd be looking to have a way to render the tree only from an array.

The problem, as I believe I've mentioned other times in similar issues, is that this library doesn't have a default "style" for the blocks (if you ignore the demo, which is meant to be a proof of concept). What I mean by this, is for example, maybe somebody wants to implement this library in such a way that blocks contain GIFs or images in them, or even custom elements such as dropdowns, text inputs, etc. Without storing the HTML, those elements would be lost in the process.

Obviously if the blocks all followed a pattern, and you were able to generate a block using only its content / ID, then you wouldn't need the HTML. But I fail to see how it would be possible to allow for all sorts of customization without the HTML - you would if anything still need to store the content of each individual block, if you didn't store the whole canvas, but I'm not sure if that's the sort of solution you're looking for?

I would be keen to implement a better method both for outputting and importing / rendering on demand, so if you have a better idea on how to go about it, I would appreciate it!

And while we are at it, I would also argue that we should NOT pass the attr together with the blocks (at least not to the render function, because render function only expects the block information and data which can be passed manually).

I believe I added this at the request of #14, as a way to allow for custom logic. Aside from that it's also helpful to recognize what element in the DOM represents a certain block in the array. I suppose if you removed the need for HTML as I explained before, then a render function wouldn't require that since it would be dissociated from the DOM.

One last thing is regarding the current api, perhaps if we switch to this it will be easier to allow easier change in the future:

flowy(canvas, ongrab, onrelease, onsnap, onrearrange, spacing_x, spacing_y);

to

flowy({canvas: element, onGrab: function, onRelease: function, onSnap: function, onRearrange: function, render: function, spacingX: number, spacingY: number});

or (only canvas is a must, all other things are optional)

flowy(canvas, {onGrab: function, onRelease: function, onSnap: function, onRearrange: function, render: function, spacingX: number, spacingY: number});

Right, I assume that is because of adding new methods which "breaks" the order of the methods passed to flowy(). I will look into it, I have to admit I am not very happy with the current initialization method - while passing the canvas is obviously necessary, the way the other optional functions are passed, successively, makes it harder to initialize it in a very custom way (e.g. only with onRelease and onRearrange methods). So I think what you're proposing makes sense.

[yellow1912]

@alyssaxuu Thank you for your quick reply.

1. Render function: Regarding the HTML part, most of the time (if not 100%) we render the blocks using certain kind of logic based on the block data. So as long as the block data is stored inside the database, re-rendering the whole tree exactly the same should not be an issue I believe. With this, it's also possible to re-render any specific block on run time (lets say certain data inside the block is changed and we can update the display accordingly).

Lets say that a block contain a gif, or text input etc. Wouldn't that depends on the data you pass when rendering a specific block (and thus the developers should also store this data for later re-rendering)

That said, we can still employ a hybrid version: pass the html but make it optional. If the html is present, we can use it as a quick way to setup the tree. That way, things work just like before for people who prefer to store the whole tree html.

2. Standardize block data structure: Right, I see that now in https://github.com/alyssaxuu/flowy/issues/14. I would argue that as long as the full draggable block data is passed, developers should not rely on an additional attr however. But that's not important, perhaps there are use cases I have not thought off.

3. Change of init API: Yup. Exactly for that reason. You can add new "options" in the future much easier.

PS: I'm working with flowy on the preset that both draggable blocks and the tree blocks are stored and queried from a relational database (which I think is true for many use cases). That is the reason why I emphasize the need of standardizing data structure and the possibility of re-rendering everything from available data.

Lets take https://tray.io/ as a very good example for the use case of flowy. The Connectors are draggable blocks in this case.

1. When we drag the blocks and drop on the tree, we know exactly which connector that is (with all the "data" of such connector) enough for us to render the connector correctly.
2. When we update the configs of any block, we know a bit more data about this block and can decide to re-render the corresponding block to update. (i.e: status change from enabled to disabled etc)
3. The connectors are already stored inside database. Each block on the tree is also stored on another table with all the necessary information.
4. When we need to re-render the whole tree, we can easily get an array of blocks for that tree, the relationship among blocks, they type (connector) of each block, the connector data and the specific block data.
5. With this, we can re-render the whole tree easily even if each block can have very specific html (gif file, input text, etc).

[alyssaxuu]

1. Render function: Regarding the HTML part, most of the time (if not 100%) we render the blocks using certain kind of logic based on the block data. So as long as the block data is stored inside the database, re-rendering the whole tree exactly the same should not be an issue I believe. With this, it's also possible to re-render any specific block on run time (lets say certain data inside the block is changed and we can update the display accordingly).

I just don't see how the blocks would be re-drawn only using the data from the array. Keep in mind this is a method that I have to create - as in, being able to redraw the whole tree with the blocks and arrows from a single array. Because of that, I need something that tells me how the blocks will be styled.

So for example if I have an array and a block contains certain data regarding the logic, how am I supposed to generate that block via a render method? Should I add a new method to individually render the blocks, so e.g. you receive the block data (for example, a specific ID + logic that is linked to a certain block type in your database and furthermore a template) and you return the HTML based on the templates you've created for your application? This is the current blocker.

[arnoldligtvoet]

I was working on some ideas regarding this as well. For me there would need to be a way to store and retrieve flows to and from a database, also it would be nice to separate structure and design.

This afternoon I had a spare hour and created the following:

• option to read files in the html page (ugly and temporary)
• expanded the file with a json array that contains the info to re-create the blocks
• expanded the flowy.import function, so it can still import the html array and process that, but also import the new json array and process that
• for processing and redrawing the canvas I created a test function drawCanvas that loops through the array, creates the html, replaces innerHtml (similar to the orginal import function) and then the rearrangeMe() can be called similar to the current import.

Just tested now with a simple flow (only three blocks), but it seems to work. Draws the flow and arrows.

It would offer the option to perhaps export as you do currently or in json and also import in html or json, making sure that the library is still simple to use (without needing a database).

I would like to work on some more ideas:

• when i was done with the test I noticed that most info is already in the blocks json array, so i could also consider integrating my json array in this one
• solution needs some kind of template system
• also would like to modify export so it exports in html (as current) or in workable json format
• also would like to integrate the additional parameters in the right pane into the json info

Can share my changes if that is of interest.

[yellow1912]

@arnoldligtvoet please do, always good to see how it is done.

[yellow1912]

@alyssaxuu

This is how I see it: we always have 2 types of data:

1. The "draggable blocks" data. This data is useful mostly for generating the draggable blocks but can also be used inside the render function (for example, an and condition draggable block may be shown differently)
2. The "tree blocks" data. This data reflected the dragged, configured blocks. It show how blocks in the tree are connected, and how each block is configured, and it also stores a reference to the corresponding draggable blocks. This data is also necessary inside the render function.

When we import the tree, both data can be passed to the import, or alternatively the "tree blocks" data should contain the corresponding "draggable data".

Lets take your demo code for example: https://github.com/alyssaxuu/flowy/blob/master/demo/main.js

On your demo, you have 3 main sections: the left draggable menu, the center flow tree, and the right configuration menu. Lets ignore the left and the right as they are not important for now.

Everything inside the snapping function relies on "value" which is essential the data of the "draggable blocks".

The other functions are really related to the rendering of the tree as I see it. If you do have an example when the above data is not enough for us to redraw the tree please let me know. Perhaps I missed something?

[fcnyp]

If exporting without HTML it should be added as an optional flag so it will be backwards compatible

[yellow1912]

@fcnyp certainly, it should be easy to check for the present of HTML option.

[yellow1912]

While flowy does not specify how data should be stored, lets expand this topic a bit on data storage just to explain how the data can be used to later re-build the tree.

Table Connector This table stores all the "draggable blocks" with the following essential columns:

1. ID
2. Name
3. Summary
4. Form (the information render corresponding configuration form on the right side)

(Connector table has enough information to render the left and right side of the demo)

Table Flow This tables stores all the "dragged blocks" configuration with the following essential columns:

1. ID
2. ConnectorId
3. ParentId
4. RootId (optional)
5. Configurations

Our interest is re-rendering a flow tree. With a given RootId (or just ParentId), we can easily query all the blocks from Flow table and rebuild the son structure to pass to flowy:

[
  {
    id: 1,
    parent: 0,
    data: {}
  },
  {
    id: 2,
    parent: 1,
    data: {}
  },
  {
    id: 3,
    parent: 2,
    data: {}
  },
]

The data is whatever the render function needs to render a specific node on the tree:

Data may contain:

1. ConnectorId to know which icon to use
2. The ConnectorName to use as the name on the node etc...

The developer who uses flowy is responsible for knowing exactly which data he needs to re-render a node and to process and pass the node tree to the import function.

Alternatively, we can even decide to separate the Connectors and the Flow data, so with flowy import we have something like this:

{
  "connectors": [
    {
      "id": 1,
      "name": "New visitor",
      "data": {}
    },
    {
      "id": 2,
      "name": "Action is performed",
      "data": {}
    }
  ],
  "blocks": [
    {
      "id": 1,
      "parent": 0,
      "connector": 1,
      "data": {}
    },
    {
      "id": 2,
      "parent": 1,
      "connector": 2,
      "data": {}
    },
    {
      "id": 3,
      "parent": 2,
      "connector": 2,
      "data": {}
    }
  ]
}

So now you can see that

1. We can initialize flowy with both Connectors and Flow blocks (and can import on the fly as well).
2. We can easily re-render any tree because we have enough data
3. We can avoid passing too much information via the attributes. If we initialize Connectors like above, on the attribute we only have to mark the ids of the Connectors.

[alyssaxuu]

While flowy does not specify how data should be stored, lets expand this topic a bit on data storage just to explain how the data can be used to later re-build the tree.

Table Connector This table stores all the "draggable blocks" with the following essential columns:

1. ID
2. Name
3. Summary
4. Form (the information render corresponding configuration form on the right side)

(Connector table has enough information to render the left and right side of the demo)

Table Flow This tables stores all the "dragged blocks" configuration with the following essential columns:

1. ID
2. ConnectorId
3. ParentId
4. RootId (optional)
5. Configurations

Our interest is re-rendering a flow tree. With a given RootId (or just ParentId), we can easily query all the blocks from Flow table and rebuild the son structure to pass to flowy:

[
  {
    id: 1,
    parent: 0,
    data: {}
  },
  {
    id: 2,
    parent: 1,
    data: {}
  },
  {
    id: 3,
    parent: 2,
    data: {}
  },
]

The data is whatever the render function needs to render a specific node on the tree:

Data may contain:

1. ConnectorId to know which icon to use
2. The ConnectorName to use as the name on the node etc...

Alternatively, we can even decide to separate the Connectors and the Flow data, so with flowy import we have something like this:

{
  "connectors": [
    {
      "id": 1,
      "name": "New visitor",
      "data": {}
    },
    {
      "id": 2,
      "name": "Action is performed",
      "data": {}
    }
  ],
  "blocks": [
    {
      "id": 1,
      "parent": 0,
      "connector": 1,
      "data": {}
    },
    {
      "id": 2,
      "parent": 1,
      "connector": 2,
      "data": {}
    },
    {
      "id": 3,
      "parent": 2,
      "connector": 2,
      "data": {}
    }
  ]
}

So now you can see that

1. We can initialize flowy with both Connectors and Flow blocks (and can import on the fly as well).
2. We can easily re-render any tree because we have enough data
3. We can avoid passing too much information via the attributes. If we initialize Connectors like above, on the attribute we only have to mark the ids of the Connectors.

Yes, this makes sense. Like I said, in that case, there would need to be a "block rendering" method for each individual block to be rendered with the data (e.g. to show a certain icon, text, inputs...) to allow for customization at the same time that the tree structure (arrows / positioning) is rendered. Just thinking what would be the most intuitive way to implement that.

Obviously I could create a method that could generate the blocks as shown in the demo (same design) using the data (icon type, title, summary...), but that has almost 0 customizability.

[yellow1912]

@alyssaxuu I think the burden of the rendering is on the developer who uses this library. The most difficult part for me as a developer was to draw the tree with all the svg connectors etc to be honest.

Going back to your demo, I think you have lots of if/else to render the block anyway: https://github.com/alyssaxuu/flowy/blob/master/demo/main.js

The only different now, is to move all that to a render function. Lets assume that we have this render function:

render(connectorData, blockData = {}, parentConnectorData = {}, parentBlockData = {}) {
    // only the first data is essential to render the node
   if (connectorData.id == 1) {
return "<div class='blockyleft'><img src='assets/eyeblue.svg'><p class='blockyname'>New visitor</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>When a <span>new visitor</span> goes to <span>Site 1</span></div>";
} else if (connectorData.id == 2) {
return "<div class='blockyleft'><img src='assets/actionblue.svg'><p class='blockyname'>Action is performed</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>When <span>Action 1</span> is performed</div>";
} else if (connectorData.id == 3) {
return "<div class='blockyleft'><img src='assets/timeblue.svg'><p class='blockyname'>Time has passed</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>When <span>10 seconds</span> have passed</div>";
} else if (connectorData.id == 4) {
return "<div class='blockyleft'><img src='assets/errorblue.svg'><p class='blockyname'>Error prompt</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>When <span>Error 1</span> is triggered</div>";
} else if (connectorData.id == 5) {
return "<div class='blockyleft'><img src='assets/databaseorange.svg'><p class='blockyname'>New database entry</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Add <span>Data object</span> to <span>Database 1</span></div>";
} else if (connectorData.id == 6) {
return "<div class='blockyleft'><img src='assets/databaseorange.svg'><p class='blockyname'>Update database</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Update <span>Database 1</span></div>";
} else if (connectorData.id == 7) {
return "<div class='blockyleft'><img src='assets/actionorange.svg'><p class='blockyname'>Perform an action</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Perform <span>Action 1</span></div>";
} else if (connectorData.id == 8) {
return "<div class='blockyleft'><img src='assets/twitterorange.svg'><p class='blockyname'>Make a tweet</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Tweet <span>Query 1</span> with the account <span>@alyssaxuu</span></div>";
} else if (connectorData.id == 9) {
return "<div class='blockyleft'><img src='assets/logred.svg'><p class='blockyname'>Add new log entry</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Add new <span>success</span> log entry</div>";
} else if (connectorData.id == 10) {
return "<div class='blockyleft'><img src='assets/logred.svg'><p class='blockyname'>Update logs</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Edit <span>Log Entry 1</span></div>";
} else if (connectorData.id == 11) {
return "<div class='blockyleft'><img src='assets/errorred.svg'><p class='blockyname'>Prompt an error</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>Trigger <span>Error 1</span></div>";
}
}

[yellow1912]

In fact, we can be smart and make it even shorter like this:

render(connectorData, blockData = {}, parentConnectorData = {}, parentBlockData = {}) {
    return "<div class='blockyleft'><img src='" + icons[connectorData.id] + "'><p class='blockyname'>" + connectorData.name + "</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>" + connectorData.summary + "</div>";
}

[alyssaxuu]

In fact, we can be smart and make it even shorter like this:

render(connectorData, blockData = {}, parentConnectorData = {}, parentBlockData = {}) {
    return "<div class='blockyleft'><img src='" + icons[connectorData.id] + "'><p class='blockyname'>" + connectorData.name + "</p></div><div class='blockyright'><img src='assets/more.svg'></div><div class='blockydiv'></div><div class='blockyinfo'>" + connectorData.summary + "</div>";
}

I see. So as I understand it, the developer would use a "render" method (not what you're describing, something like flowy.render(blocks)) to pass an array of blocks first, and then they would use a callback (which I assume is what you're describing) that would be send back the information of each block individually with all of its data, so the developer can then return the HTML.

Seems reasonable enough. I assume since this would allow for the blocks to be different dimensions than they originally were, the entire tree would have to be recalculated as well, so that would allow flexibility in regards to changing the design of a product while still being compatible with old data.

I think it is pretty straightforward now - I will definitely work on implementing something like this.

[yellow1912]

@alyssaxuu That sounds great.

If I may, I would like to suggest the following approach:

1. Changes to the initialize method: flowy(canvas, connectors, render, {onGrab: function, onRelease: function, onSnap: function, onRearrange: function, spacingX: number, spacingY: number})

This is a BREAKING CHANGE, but I think it's necessary because: a. optional configurations, callbacks can be passed in a much more dynamic way b. canvas is essential to render the tree. I would argue that connectors array and render function is also essential, but that is not necessary, can be put inside the options as well.

2. Changes to the way how a node is rendered: Whenever a node is added to the tree (can be via drag-drop, can be via import), we shall callback the render function to render that specific node. Here we should note that: a. this render function is responsible for rendering the node only, not the whole tree (so in the function argument we only pass the node data, not the whole tree); b. this render function is called regardless of how the node is added (drag n drop or manual import)

3. Changes to the data we handle the import method: The import method will be our way to programmatically add new blocks or connectors (I think we should require initializing and importing connectors as per my comment above to avoid relying on the attributes of the draggable elements). Everytime a new block is added, we callback the render function to render that specific block/node

4. Add an remove/delete method for node/block: This helps with other issues, for example we can now easily add a trash icon or handle it however we want to delete a node/block

5. Add onAfterRender and onBeforeDestroy callbacks for node/block: This is optional, but would be cool. Lets say on the onAfterRender you can add a listener to a trash icon inside the block to call delete. On the onBeforeDestroy perhaps you want to manually remove that listener.

[yellow1912]

@alyssaxuu if you need any additional hand to work on the new changes please do let us know. We can create a new branch to work on the new features to keep the master branch stable. I look forward to the changes

[yellow1912]

@alyssaxuu I found this library which can give us many ideas:

https://github.com/kieler/elkjs

The tree structure is well thought-out:

const graph = {
  id: "root",
  layoutOptions: { 'elk.algorithm': 'layered' },
  children: [
    { id: "n1", width: 30, height: 30 },
    { id: "n2", width: 30, height: 30 },
    { id: "n3", width: 30, height: 30 }
  ],
  edges: [
    { id: "e1", sources: [ "n1" ], targets: [ "n2" ] },
    { id: "e2", sources: [ "n1" ], targets: [ "n3" ] }
  ]
}

I would convert it to:

const graph = {
  id: "root",
  layoutOptions: { 'elk.algorithm': 'layered' },
  children: [
    { id: "n1", data: {} },
    { id: "n2", data: {} },
    { id: "n3", data: {} }
  ],
  edges: [
    { id: "e1", sources: [ "n1" ], targets: [ "n2" ] },
    { id: "e2", sources: [ "n1" ], targets: [ "n3" ] }
  ]
}

Also, the rendering is done via a worker (neat, not necessary for now but is super neat for future performance improvement)

[GitStorageOne]

The problem, as I believe I've mentioned other times in similar issues, is that this library doesn't have a default "style" for the blocks

Maybe would be useful to define block constructor function as input parameter.
When you need to draw item, would be possible to just call that function and pass useful block metadata.
Also, as an additional suggestion, switching to Typescript will add many benefits to this library.

interface IBlock {
  id: number; // or even better - string, to store UUID.
  // other block data
}

flowy(canvas, { blockConstructor: (block: IBlock) => 'block html' });

[yellow1912]

The problem, as I believe I've mentioned other times in similar issues, is that this library doesn't have a default "style" for the blocks

Maybe would be useful to define block constructor function as input parameter. When you need to draw item, would be possible to just call that function and pass useful block metadata. Also, as an additional suggestion, switching to Typescript will add many benefits to this library.

interface IBlock {
  id: number; // or even better - string, to store UUID.
  // other block data
}

flowy(canvas, { blockConstructor: (block: IBlock) => 'block html' });

Would be nice, but not 100% mandatory. You can get around the issue with javascript, albeit not the best looking code but still work.

[wscjxky]

may be you can try. https://github.com/alyssaxuu/flowy/issues/148