Where to next? (Compilers vs Interpreters)

[mattfysh]

Hi there. I'm currently writing a web scraping DSL and was excited to come by this project. have poured through the docs and fairly happy with the AST being produced.

I understand the next step is to write a compiler or interpreter to make the AST "executable", but I'm a little unsure where to find material on bridging AST and program execution, e.g. should I make a compiler or an interpreter, and how would I build one in javascript?

Thanks!

[lorefnon]

For this kind of use case, IMHO the easiest approach is to target javascript and leverage the babel ecosystem for generating it.

Instead of creating your own AST format, you can use the @babel/types package to generate babel compatible AST in your post-processors, and then use @babel/generator to generate javascript from it.

[mattfysh]

Thanks @lorefnon 🙏 I ended up writing an interpreter in JavaScript - which actually turned out to be a lot of fun, and much less code than I was anticipating! It all hinges on an evalNode function which takes an AST node and runs it through a big switch statement to determine which type-specific eval function to call.

One of the coolest things I realized is how easy it can be to change the way an AST is evaluated when executed, which led to a really neat feature where some of the loops in my programming language can be evaluated in parallel. Next thing I'm looking into is attaching a debugger and building some kind of UI for writing, executing and debugging the code in realtime :)

[lorefnon]

Oh cool. Will this be an open source project ?

[sibelius]

I would love to see some article on this

[mattfysh]

I'm hoping to release it as an open source project soon. It's a purpose built language for web scraping called webslang. I'm currently building out a prototype to demo to investors, and then hopefully once I've secured funding I'm able to bring a few more people onboard and we can look at open sourcing the language :)

The next thing I'm hoping to look into is the use of a "Suspend" feature where the execution of the script can be interrupted and resumed, for example following pagination links when scraping.