Provide documented source code

[xkxx]

Hi Gravicappa,

I'm currently developing a patch for shen-js that provides command-line REPL based off Node.js (Essentially a Node.js port). I'm currently readying my code and hopefully you will see my pull request popping up here very soon. However, the lack of documented source code is giving me a lot of headache on debugging. The development source code (primitives.js, backend.js, etc) are missing from the repository. Currently shen.js is a ~1MB machine-generated JavaScript blob that is cryptic and impossible to understand and extend upon. As a result, I'm forced to do lots of workaround in my code in order to not touch the gigantic shen.js file. Can you please upload the development source code onto github so that we can collaborate better on this project? Thanks!

Also a quick question. Does shen-js currently support compiling standalone JavaScript that can be executed without the interpreter?

[gravicappa]

Hello.

The development source code (primitives.js, backend.js, etc) are missing from the repository.

All files that are required to build shen.js are either in this repo or in its dependencies (js-kl, reg-kl, shen-libs). File primitives.js is generated (see modules.shen).

Currently shen.js is a ~1MB machine-generated JavaScript blob that is cryptic and impossible to understand and extend upon. As a result, I'm forced to do lots of workaround in my code in order to not touch the gigantic shen.js file.

Can you explain what are you trying to do? The only thing that is subject to porting is I/O. And extending I/O is explained in README and implementing it does not require workarounds.

[xkxx]

All files that are required to build shen.js are either in this repo or in its dependencies (js-kl, reg-kl, shen-libs). File primitives.js is generated (see modules.shen).

Thanks, I'll look into that.

Can you explain what are you trying to do?

I'm trying to write a commom-line utility that compiles Shen to JavaScript under Node.js runtime. All I need is a simple function like Shen.compileToJS(string) and a (preferably separate) runtime library. Read along for the problems that I've encountered.

The only thing that is subject to porting is I/O. And extending I/O is explained in README and implementing it does not require workarounds.

• Shen.js does not follow the CommonJS standard and therefore can not be imported with require('./shen.js'). The functions are all over the place without a common namespace, so I can't even convert the file to CommonJS by hand. In my patch I had to set up a sandbox to execute the code in.
• Initialization is kind of awkward. There isn't a Shen.init(options) function that you can supply the i/o functions with, you have to load the i/o functions before loading shen.js, or else shen.js will try to take it upon its hands and cause errors like "write is undefined".
• Also, if shenjs_external_repl is not set, shen.js goes straight to repl mode, which is undocumented and caught me by surprise. It would be great to have Shen.init({defaultRepl: false, io_functions...})
• There isn't a simple function like Shen.eval(string); instead, shen_read_evaluate_print() takes on i/o itself. The problem is that Node.js has its own REPL module that does i/o much better than Shen, and in the browser you have to get the string from <input> anyway. Also I'm trying to find a function like Shen.compileToJS(string) so I can compile the code in the main execution context and run the code in a separate sandbox, but I haven't been able to find it yet.

[gravicappa]

Shen.js does not follow the CommonJS standard and therefore can not be imported with require('./shen.js'). The functions are all over the place without a common namespace, so I can't even convert the file to CommonJS by hand. In my patch I had to set up a sandbox to execute the code in.

I am not familiar with CommonJS (and with JS itself as well to be honest). Can you provide a sample?

Initialization is kind of awkward. There isn't a Shen.init(options) function that you can supply the i/o functions with, you have to load the i/o functions before loading shen.js, or else shen.js will try to take it upon its hands and cause errors like "write is undefined". Also, if shenjs_external_repl is not set, shen.js goes straight to repl mode, which is undocumented and caught me by surprise. It would be great to have Shen.init({defaultRepl: false, io_functions...})

I'll think about this. Does using such namespaces hurt performance?

There isn't a simple function like Shen.eval(string); instead, shen_read_evaluate_print() takes on i/o by itself.

There was no read-from-string function in previous versions of Shen. Perhaps I should move shen-js to latest Shen.

[xkxx]

I am not familiar with CommonJS (and with JS itself as well to be honest). Can you provide a sample?

According to CommonJS standard, each js file is isolated from each other. Say you have a js file named module.js,

var publicVar = 123, privateVar = 321;
exports.publicVar = publicVar;

When you call var module = require('./module'), module.js gets executed and its exports property is returned,

module.publicVar // 123
module.privateVar // undefined

Please refer to http://nodejs.org/api/all.html#all_exports for details.

I'll think about this. Does using such namespaces hurt performance?

Having namespaces is a standard practice among js libraries. Due to the way js works, I think separating functions into a unique namespace will perform better than dumping everything into global (since querying against a giant hashtable like global or window is slower than querying against a smaller one)

There was no read-from-string function in previous versions of Shen. Perhaps I should move shen-js to latest Shen.

If there is anything I can help with, please let me know.

Also, can you provide me with some documentation on shen-js and js-kl modules? What function do I need to evoke to translate a shen string into javascript?

[xkxx]

Hello? Any update on this? The latest version of Shen is now 9.1, which seems to be compatible with Shen 9.0 API/ABI-wise. If you aren't able to update shen-js in time, perhaps I should start preparing my pull request. It won't have transpile / execute support, but REPL seems to be working properly for the most part.