ReasonML: run natively, not just via BuckleScript

[raxod502]

imported comment by @kwshi

Summary

ReasonML's core language is essentially identical to OCaml, just with a different syntactic presentation. Riju currently runs ReasonML via bs-platform (BuckleScript), which is a OCaml-subset-to-Javascript transpiler, supplemented with a collection of JS-standard-library bindings (e.g. Js.log -> console.log).

Doing so restricts support for OCaml's own built-in primitives (e.g. Sys.readdir) and also makes the code run slower (since it is now running via JS, instead of native compiled code--though that's a relatively minor concern). I propose that Riju support running ReasonML natively, perhaps in addition to via BS. Doing so allows ReasonML to be powered by the latest-version OCaml runtime and gives full support for built-in primitives.

Background

At its core, ReasonML is just an alternative syntax for the OCaml language. The underlying type system, syntactic constructs, standard library, etc. are all identical; the only differences are keyword/punctuation choices. (This claim is not 100% true; recently added to ReasonML are a select few additions meant to make it resemble JS(X) even more. Nevertheless, it is still true that there is an almost perfect one-to-one correspondence between ReasonML code and OCaml code.) From the ReasonML website:

Note: Reason can be translated back and forth to OCaml using refmt. Anything possible in Reason is also possible with OCaml and vice versa.

In fact, the Try ReasonML tool demonstrates these one-to-one conversions between ReasonML and OCaml.

BuckleScript and friends

Despite its OCaml core, ReasonML's primary target audience is web-development folks who really like JS. As such, ReasonML is almost always advertised as a standalone language meant to be run via bs-platform (BuckleScript), which is an OCaml-to-JS compiler supplemented with bindings to the JS standard library. However, running OCaml/Reason via a JS transpiler reduces support for OCaml's own built-in primitives.

For instance, the Sys.readdir function (which lists children of a directory) is accessible via ReasonML but fails to run when transpiled via BS--giving a compile-time warning

Warning 106: BuckleScript warning: Unimplemented primitive used:caml_sys_read_directory

and run-time error

  /tmp/riju/02bd35ee-7ee0-46f1-84f0-9e65ea1fb112/node_modules/bs-platform/lib/js/caml_external_polyfill.js:16
    throw new Error(s + " not polyfilled by BuckleScript yet\n")
    ^

Error: caml_sys_read_directory not polyfilled by BuckleScript yet

Even though it is technically valid ReasonML. This behavior makes some sense, since JS makes a client-side/server-side distinction, and primitives such as filesystem access are only available on the server-side via NodeJS-provided libraries such as fs, os, path, etc. As such, Reason-BS users are advised to manually add external bindings to corresponding NodeJS libraries, or rely on community-provided NodeJS binding declarations, to get proper access to "server-side" primitives.

This restriction is not inherently wrong or undesirable, since ReasonML's most common use case is transpiling to JS and executing in a JS runtime. But considering that ReasonML is the same core language as OCaml, there are situations where one might prefer to run ReasonML in the native OCaml runtime instead (it is, after all, fully capable of doing so).

How to?

There are, to my knowledge, two ways to run ReasonML as native OCaml:

1. Use refmt to convert ReasonML syntax to OCaml syntax, then just call ocaml on the produced source file.

2. Use bsb-native, which, based on what I can tell, pretty much does the same thing as above.

   (Yes, the naming is really confusing to me too, in case you're wondering--it calls itself "bucklescript", even though "bucklescript", at least originally, referred to the OCaml-to-JS transpiler--to me, "bucklescript native" is a total oxymoron. I think the names are the way they are because people like to conceptualize ReasonML as a standalone language with a standalone compiler, rather than as a dialect of OCaml. Because, you know, all the hip kids dig JS, not a functional programming language most known for its ubiquity in French computer-science academia.)

My proposal is to add running natively as an option for ReasonML. Certainly, some people may still want to test BuckleScript's JS interop behavior or fiddle with Bucklescript's Belt/Js standard libraries, which are not available outside of BS. So I think it's probably best to have both options available, somehow.

Random considerations

• The README says that Riju aspires "to support more languages than any reasonable person could conceivably think is reasonable." In the case of ReasonML/OCaml/Bucklescript/JS, many things could constitute a "language":

  • Runtime (native OCaml vs. NodeJS)
  • Syntax (ReasonML vs. OCaml)
  • Libraries (native OCaml vs. BS standard library + JS bindings)

  And in fact, not much stops us from mixing-and-matching these parts--e.g., I'm pretty sure nothing stops me from writing code in plain OCaml syntax, then transpiling it (via BS) to JS. I'd be writing OCaml code but running it in an archetypally ReasonML environment. At what point should something be considered a different language (and added in Riju as such), vs. a "dialect"? If it makes sense to support native ReasonML on Riju, does it also make sense to support the vice versa, JS-run OCaml? What about OCaml's bytecode-to-JS compiler js_of_ocaml (as opposed to BS, which operates on OCaml source code rather than precompiled bytecode)?

• Similarly, what are your thoughts on Deno, the hip new runtime for TS/JS? It runs the same language as TS/JS, but conceivably has different runtime behavior. Should it be available as a separate language option, or selectable somehow in Riju? In one were to support Deno, should TS/JS even be considered separate languages, or just different configuration profiles on the same runtime?

[raxod502]

Thanks for the detailed analysis; it's extremely helpful in outlining the landscape, which makes it a lot easier to add support (half of the difficulty in adding languages is untangling how they are supposed to work). To respond to specific points:

• So far on Riju, "language" has referred to syntax, and I've avoided adding more than one version, implementation, dialect, or runtime of the same syntactic language unless the dialects are so different that they seem like different syntactic languages (the last part being a subjective judgment).
• I have no inherent objection to changing this policy in the future. So far I chose it just for the sake of perceived cleanliness. If we were to add multiple implementations, versions, etc. of the same language to Riju in the future, I'd like to represent this in the data model somehow. For example, if we were to have Python 2 and Python 3, I'd prefer for those to be two different options under the Python language, rather than two separate languages (which is what some other similar services do). The same would be true for ReasonML and its multiple implementations.
• Until we get to the point of having multiple implementations per language, I think using the native runtime rather than the JavaScript-transpiled version would be better, as you suggested.