Abell AST explorations - Githubissues

pantharshit00 commented 4 years ago

( I am going to do a thought dump here, I can also see us writing a tech blog about this. Basically I will start working after I other abell team members approves )

Right now we basically do string modifications to an abell file. We extract out the contents of {{ }} which are essentially regions of javascript inside of your html, which may or may no return a value. That is determined by the execution of that javascript in a V8 isolate provided by Node(https://nodejs.org/api/vm.html).

This has worked pretty fine for now. But this approach makes it very hard to add features to abell especially features which require us to add additional metadata to the abell tree. Doing string replacements on a whole html tree is very error prone and it will be a unmaintainable mess for us. This also debars us from adding error messages to abell when a abell component is invalid. Right now it will just generate rubbish html.

For example, right now no one will stop you from doing this

<AbellComponent>
{{ const something = "this"}}
<template>
    <h1>{{ something }} }}</h1>
</template>
</AbellComponent>

Guess the output of the above template 😛. IMO it should be an error instead with a proper stack trace.

Now the solution here is what almost every compiler or template engine does. We will need to convert abell source code into a data structure which has all of the necessary information about the source code. This process is generally known as parsing. The DS should have enough information for us to implement a “printer” for that data structure which will convert it back to a source code.

Then what we can do is transform the data structure into our need. That process is far easier than relying on string transforms. At the end we can just print the result of transform. If we went with approach that I am going to suggest below, we should to perform the script execution before the printing step.

This data structure is typically called an Abstract Syntax Tree which represents the code in form of a tree data structure and it omits details like punctuations to make it “abstract”. For example, we don’t need to keep the < token in order to know it is a tag. That detail is abstracted out of parse tree. We will need to design an AST for abell which will enable to us all of the above mentioned things.

Now we come to the real question, what it should look like, should we just do the current replacements and then just parse the html and css syntax using an existing parser? Is this even worth the effort? Will this make abell “strict”? Argh?!

Approach 1

Here is the first thought that came to my mind. Use htmlparser2 + stylis for the CSS. We do the replacements, then these tools can parse the data we pipe into them and do the transformations. This is a very good naive approach at first. Much less work and we gain.

Pros

Fast to implement as most of the work is already done by these libs
No need to maintain a parser

Cons

htmlparser2 doesn’t come with a printer. It is strict parsing library, so we would have need to figure out how to results from it.
No way to embed abell tags {{ }} in the source tree. They are just replacements in the pre parsing step.
It adds dependency on two libraries

Approach 2

Now the next approach that comes in the mind is going all custom. Doing all the parsing the from scratch. Implementing all lexing, parsing and printing steps from scratch. This will give us full control.

Pros

It will give us full control
Abell renderer will remain zero dep

Cons

We will need to maintain it (bittersweet)
It will need a lot of work and we will need to make it fast
Doing CSS parsing would be actually moderately challenging as there are a lot of selectors
It will increase the time we will need to implement

Approach 3

You know it’s coming. It’s always the hybrid approach. The reasoning in this approach is that since most of the abell syntax is baked into the html part, we can write a parser from scratch for that to have full control there. We can have stylis to parse the CSS for us. It will also give us additional features which stylis provides.

Props

It will still give us control over the abell part. We will be able to add more syntax features in the future
It will cut scope for the work need to done for the CSS part.
CSS will be parsed tree which we will embed inside of parsed tag from our abell. We can apply our transformations there.
It will still give us stylis features like & reference

I will suggest we go with approach 3. That sounds most promising to me. If you all agree I can start working on it and I will writing a brief design proposal of the AST here.

Also, all this means almost a rewrite of abell-renderer 😅.

With this, abell will be like a superset of html with some additional features.