node.js embedding API

[nasser]

A major unaddressed usecase for ClojureScript is to host it inside of larger node.js projects. I do this all the time, namely in Electron applications, where it can act as the scripting or live coding layer for the underlying JavaScript. I built and maintain the clojurescript npm package that does exactly that and exposes an embedding API.

lumo is much better architected, more up to date, and better maintained than my package. Ideally, lumo would be the base of this embedding API so that we're not duplicating work. With some hacking of the build scripts (using "src/js/cljs.js" as the input to rollup) I was able to get it to work, too.

$ node
> eval(fs.readFileSync("target/main.js", "utf8"))
[Function: accessor]
> var embeddedLumo = require("./target/cljsBundle.min")
undefined
> embeddedLumo.default({verbose:false,scripts:[]})
Promise { ... }
> embeddedLumo.execute("(+ 1 2)")
3
null
> embeddedLumo.execute("(defn working [x] (str \"its \" x \" working!\"))")
#'cljs.user/working
null
> embeddedLumo.execute("(working :totally)")
"its :totally working!"
null
> cljs.user.working("totally")
'its totally working!'
> embeddedLumo.execute("(ns other.core)")
nil
null
> embeddedLumo.execute("(defn even-namespaces [] :work)")
#'other.core/even-namespaces
null
> other.core.even_namespaces()
{ ns: null,
  name: 'work',
  fqn: 'work',
  _hash: 385770312,
  'cljs$lang$protocol_mask$partition0$': 2153775105,
  'cljs$lang$protocol_mask$partition1$': 4096 }

With some work, a contextified embedding API on top of lumo is totally possible.

My question is the following: does an embedding API make sense as a part of lumo, or is another package a better place? If this is something that lumo wants, I am more than happy to put together a PR. In that case, I would also deprecate my package and tell people to use the single, unified ClojureScript environment/embed API in lumo.

If lumo is set on being a REPL/ClojureScript environment and exposing an embedding API is not a goal, I could rebuild the clojurescript package out of lumo.

I am fine with either.

[arichiardi]

I often found myself in need of a repl in, say TypeScript, and found that it is really really not the same as what lumo provides. At some point I was thinking of calling the compiler for inside lumo and transpire typescript to JS so that I could evaluate it in lumo...crazy idea I know out of frustration, sorry maybe this is not even the place for it

[shawwn]

+1. This would be a step toward a ClojureScript webpack loader.

[arichiardi]

@nasser are you still looking into doing this?

[4mitch]

Finally I found a way to run CLSJ from node! Cause I was finding a way to play-and-share with it in all this JS-Node playgrounds like CodeSandBox and RunKit etc. And even great idea to use lumo for it! It's pity that there was no response at all :( Dear @nasser, I hope you have done second option of your proposal "could rebuild the clojurescript package out of lumo"?

[nasser]

@4mitch @arichiardi I haven't had a chance to revisit this for about a year, unfortunately. My focus has been on the CLR and away from JS. I someone else is willing to take up the project I am happy to help with the transition!

[arichiardi]

@nasser that makes sense, great work on the CLR side!

About my take on this, I kind of revised my ideas around the embedding API as ClojureScript brings with it a lot of code for the persistent data structures. We code in a different way in TypeScript anyways unfortunately..

[PabloReszczynski]

@arichiardi Any news on this? Anything you need help with?

[arichiardi]

@PabloReszczynski We decided to switch to ClojureScript vanilla (using shadow-cljs) in our current code base so I am not actively trying to solve this problem at the moment.

[djblue]

In my attempt to implement an nrepl server for lumo, I found the best way to integrate is via execute-text. Originally, I was going to use plain eval but it turns out that isn't enough. I took a deeper dive into lumo and found repl.cljs which sets up all the clojurescript plumbing and adds a lot of useful repl features.

The main issue I ran into is now I have to use js/$$LUMO_GLOBALS.doPrint to capture values because I'm leveraging internal implementation details.

Does anyone have any ideas on what an embedding API should look like? I would be willing to implement such an API for lumo but wanted to get buy-in before going off and implementing something.