How does this compare to https://github.com/gravicappa/shen-js

[Gozala]

Hi

I'm just curious how does this compares to https://github.com/gravicappa/shen-js or how complete current implementation is.

Thanks

[rkoeninger]

@Gozala added this to the readme:

This implementation is not complete. It can almost run the Shen test suite, but fails on some of the more intensive tests with a stack overflow. ShenScript uses trampolines for tail recursion, but it's still not enough. Only Safari has implemented tail calls natively at this point. ShenScript also can't simulate synchronous I/O, which the read-byte primitive is designed for. So the built-in (shen) REPL won't work.

ShenScript was an attempt to improve on the existing shen-js project, as shen-js is more complicated, generates inscrutible code and outputs a large .js file (~12MB uncompressed). On the other hand, shen-js actually works. shen-js implements its own KLVM on top of JS, allowing it to handle deep recursion and simulate synchronous I/O. Given the current state of JS engines and the design of Shen, shen-js is the only 100% complete solution.

Despite being only 90% of the way to completion, I think ShenScript can still be useful so long as you don't want to run the REPL or your computation doesn't push the limits of recursion or performance. Typical JS development should be doable, but only beta testing will show for sure.

[rkoeninger]

@Gozala Close if you feel your curiosity has been satisfied.

[Gozala]

@rkoeninger Thanks for detailed answer, I'll followup with few more questions here if that's ok as I don't know if there is any better place to have discussion.

This implementation is not complete. It can almost run the Shen test suite, but fails on some of the more intensive tests with a stack overflow. ShenScript uses trampolines for tail recursion, but it's still not enough.

That sounds odd, could it be that some functions don't get trampolines or maybe implementation used does not handle mutually recursive case ? Otherwise I don't really see how you'd you'd get stack overflow even with trampolines. I tried to look through code but was unable to quite get what's going on.

ShenScript also can't simulate synchronous I/O, which the read-byte primitive is designed for. So the built-in (shen) REPL won't work.

I assume you're thinking of browser case, because in node you could use synchronous IO to overcome that limitations. In browsers you could also do that by running in web workers and doing sync IO from there.

Despite being only 90% of the way to completion, I think ShenScript can still be useful so long as you don't want to run the REPL or your computation doesn't push the limits of recursion or performance. Typical JS development should be doable, but only beta testing will show for sure.

Could you write some instructions of how to take simple hello world shen program and then generate JS out of it ? Also some examples of JS interop would be really great.

I'd be interested in giving it a try.

P.S. I did not have much success with shen-js even compiling examples in the repo did not seemed to work.

[rkoeninger]

@Gozala Trampolines only help when it's a tail call, so maybe there's a long chain of non-tail calls in whichever test was failing. My memory is spotty at this point.

I'm not familiar with web workers, but if you have an idea how they could be used to do sync IO, I'll try to implement it.

If you want to try it out, you can build it using the instructions in the readme. There's a translate button on the index.html page that will output javascript. Here's the section of the transpiler that does interop. Putting the js. prefix on a function outputs that function as-is without translating its name and writting a form like (js. "jscode") outputs jscode as a literal in-line in the output javascript string.

shen-js has an online hosted interactive prompt with working repl here.

[Gozala]

@Gozala Trampolines only help when it's a tail call, so maybe there's a long chain of non-tail calls in whichever test was failing. My memory is spotty at this point.

But that is also the case with JS tail call optimization.

I'm not familiar with web workers, but if you have an idea how they could be used to do sync IO, I'll try to implement it.

I guess it depends on what IO you have in mind. I was assuming you needed sync IO to read files which in browsers would probably mean loading it from server. That could be done with sync XHR in the web worker without blocking the UI thread.

If you meant reading from STDIN that is probably still doable by also adding Service Worker into the mix. Idea would be that worker would request sync XHR that will end up creating FetchEvent in service worker that can respond with a promise, which will resolve with request body of the other request comping from the UI thread.

One major limitation is you won't have access to DOM in the web workers, but I presume most shen programs that exist do not either so for REPL it might be fine.

If you want to try it out, you can build it using the instructions in the readme. There's a translate button on the index.html page that will output javascript. Here's the section of the transpiler that does interop. Putting the js. prefix on a function outputs that function as-is without translating its name and writting a form like (js. "jscode") outputs jscode as a literal in-line in the output javascript string.

Thanks, I'll give it a try.

shen-js has an online hosted interactive prompt with working repl here.

Yeah I was able to use hosted repl but was unable to compile anything on my machine due to https://github.com/gravicappa/shen-js/issues/13 which just got fixed :)