Improve the HTML parser implementation

[atulbhatt-system32]

When we load our project in rnbw and pick a html file (or index.html by default as initial file), it is parsed to create a HTML nodetree.

But while we create this node tree, we don't have access to monaco instance as it is not loaded by that time. Due to this reason, we first create the tree and add start and end index for each node provided by parser to each node data.

Now, when everytime we select a node we have to do calculation based on start and endIndex of the code segment belonging to a particular node to find out where does our code for a particular node lies in codeview(editor).

In this issue we focus on resolving this and get a tree that already has data based on where our code is in the vs code, while we build the tree.

This has 2 benefits:

• No need to calculate position of code everytime we click on a node.
• And when we will be doing any changes that leads to change in the start and endindex of a node we can directly do manipulation in the positions of the node in codeview.

Currently, we have two methods to parse the HTML code. One that parses the entire file, and the other that parses the part of the HTML code that changes.

In this we also aim to make them one.

Added Bonus: Less code, Less Bugs.

PS: It will help in zero-sync as for codeview we are solving the position calculation of nodes in the codeview when they change. And it's related to the position part of it.

[atulbhatt-system32]

Today I tried multiple approaches to find a good technique that can help us achieve codeview sync. Here are following steps I did:

• I cloned phcode.dev repo and digged in it to find how they do it. My observations are: They have their custom parser, tokenizer, and they also have their custom dom differ

• We are already sorted out with parser but we don't have a proper diffing solution. The current implementation that we are improving on which is based on the previous implementation isn't doing the diffing but rather using the concept of focused node (which doesn't exist on phcode) to determin where to perform changes.

• There are differences in the phcode works and rnbw works is because:

  • We have a treeview to maintain and also a focused node.
  • We can edit content through stage.
  • We sync the nodes so we need position data with us.

• Based on this, today I tried to achieve updating the stage with parsing entire code, which has to be done nevertheless even for diffing. The important part is updating the dom based on changes for better performance. I tried to use innerHTML to prevent rerender of stage but it had 2 issues:

  • It retains no information of where it was focused and other such event related info.
  • Web components are not rendered with their template data
  • I also tried to find the solution to load those data dynamically but it didn't worked.

• Next I tried to use some diffing libraries.

  • I tried morphDom but it's no longer maintained it seems. I tried using it but it was giving some error. Maybe the iframe from shadow dom of stage, and the new dom parsed from the updated content is not compatible.
  • There were other similar libraries too. I tired microMorph which is updated. It gave some result based on diffing. But I have yet to understand how to apply them and how useful they are.

What is diifing? Diffing gives us information about what exact part of the code has to be changed to convert an existing DOM into new DOM

[atulbhatt-system32]

Finally, the diffing has been implemented and codeview sync is great. There are some other issues which this change has brought in, or which can now be checked as the code changes from codeview are now reflected.