Duktape Module Loading Support

[ngmaloney]

I'm attempting to take a stab at creating a Typescript shard. I'm getting hung up on whether I need the ability to reference ts as a module. I'm currently trying something like the following (as a POC)

It is returning the following error:

ReferenceError: identifier 'require' undefined
Unhandled exception: ReferenceError: identifier 'require' undefined (Duktape::ReferenceError)
  from lib/duktape/src/duktape/api/error_handling.cr:48:16 in 'raise_error'
  from lib/duktape/src/duktape/api/eval.cr:122:7 in 'eval_string!'
  from lib/duktape/src/duktape/api/eval.cr:32:7 in 'eval!'
  from src/callahan-ts.cr:15:9 in 'compile'
  from src/callahan-ts.cr:22:9 in 'compile'
  from test.cr:3:1 in '__crystal_main'
  from /usr/share/crystal/src/crystal/main.cr:97:5 in 'main_user_code'
  from /usr/share/crystal/src/crystal/main.cr:86:7 in 'main'
  from /usr/share/crystal/src/crystal/main.cr:106:3 in 'main'
  from __libc_start_main
  from _start
  from ???

I started poking around the duktape docs and they mention Duktape 2.0 requires the module loading code to be included in the build. I didn't see any reference to duk_module_duktape in the duktape.cr source code. I'd be interested in PR'ing a patch but want to make sure this is the correct approach.

[jessedoyle]

Hi @ngmaloney - thanks for using duktape.cr!

A typescript shard would be really useful and sounds like a fun project! You're definitely correct on your research with regards to modules and the Duktape engine.

Duktape 2.X removed the built-in module loading framework and moved the code into an extra that can be introduced into the build. duktape.cr does not currently build the module-duktape extra.

Another compounding factor is that Duktape::Sandbox will undefine any global require functions to ensure a secure sandbox environment by default. If you use an instance of Duktape::Context, this process will not be applied before evaluation.

As the global require() function is no longer defined in Duktape 2.X, undefining a global require object is not actually doing anything today - it's just something to note.

I'd be open to any pull requests that would re-introduce the module-duktape extra into the build for this shard, but that may prove to be tricky to implement and configure due to the low-level requirements for the build.

An alternative approach is that the C module resolver code can be ported to Crystal! This would give users of this shard the ability to "enable" the module resolver when required in Crystal code without having to recompile all of C code for Duktape. This is actually an approach I investigated a while back in the require-node branch.

Of course if you have the option to shell out to node, then you can always transpile locally and pass the bundled package directly to duktape.cr. It looks like that's how the typescript-rails gem works.

Finally, if anyone does want to go down the path of writing a Crystal-based module resolver for duktape.cr, here's a high-level overview:

1. Port the C code for the module-node Duktape extra to Crystal. I tried that a while back here.
2. Write a file resolver in Crystal and expose it into the Duktape engine as defined here (this can be done via the push_proc method).
3. Implement the node module resolution algorithm in Crystal. I tried that here but didn't get very far at all.

TLDR

duktape.cr does not currently support module resolution. Shelling out to node would be the easiest solution if your shard can support that. More comprehensive solutions are definitely possible if anyone wants to submit a PR!

[ngmaloney]

@jessedoyle thank you for the detailed response. My end goal is to create an experimental middleware that bundles ts/js/css assets without any dependencies on Node. I'll do some more research based on the resources mentioned in your reply and see where that takes me. I'll close out this issue, appreciate all your work on this.

[jessedoyle]

My end goal is to create an experimental middleware that bundles ts/js/css assets without any dependencies on Node.

I'm fairly certain that's possible - my response above should help provide some direction for sure. If you find that you need any changes in duktape.cr to facilitate your development, please feel free to submit issues or PRs!