Investigate Alternative JS Engine

xu-cheng commented 3 years ago

This crate requires an embedded JS engine to execute katex. Currently, this is done using quickjs and its rust binding. However, it has certain shortages:

No support on native windows using MSVC. (https://github.com/theduke/quickjs-rs/issues/72)
No support on WASM environment. (https://github.com/theduke/quickjs-rs/issues/73)
The development activity of quickjs is low.

As such, we may want to find an alternative JS engine. Since this crate never exposes the types from the underlying JS engine, changing it will not create any breaking changes. Ideally, the alternatives need supporting all the current used features, addressing the above problems, being small to be embedded, and being well maintained.

Some candidates:

boa, a JS engine written in pure Rust. But currently, it is experimental with many basic JS features not being implemented.
rusty_v8, Rust bindings to V8 engine. This engine may be too big to be embedded.
rust-mozjs, JS engine from servo. It doesn’t seem to be well maintained. Its crate hasn’t been updated for 2 years.

xu-cheng commented 3 years ago

With v0.4.0 of KaTeX, we start to abstract the JS engine behind a common trait. Help is needed to add more JS engine backend in addition to the QuickJS.

chipsenkbeil commented 3 years ago

@xu-cheng, from the candidates you described, it doesn't seem like there's a good option as an alternative in the near future, right? The wasm issue also mentions https://github.com/theduke/slimjs, but that seems more like an experiment than anything with a future.

I was looking at incorporating your crate (https://github.com/chipsenkbeil/vimwiki-rs/issues/109) to support server-side rendering alternative to MathJax.

I've also got vimwiki-wasm to enable parsing the vimwiki language (wiki dialect with similar purpose to markdown) and searching it directly in the browser. It's not a dealbreaker to me that this won't work with wasm right now, but curious what your thoughts are for the future.

xu-cheng commented 3 years ago

Just as I mentioned above, my thought is to provide multiple JS backends in this crate. Therefore, the users can choose the suitable backend to their needs by assigning the corresponding feature to this crate.

I am planning to add two new backends for now: rusty_v8 and wasm-binding (i.e., using the JS from the browser directly). But I don't have too much free time for now. So PR is welcome. Also, it is worth mentioning that the common JS engine trait is not fixed and can be changed when we add new JS backend.

In long term, I hope we can add boa and slimjs support.

chipsenkbeil commented 3 years ago

@xu-cheng, I wonder if it's possible to embed wasmer into this crate and use it to load a QuickJS wasm file. In wasmer readme's quickstart, it specifically shows loading a QuickJS wasm file and using it via the cli. If you can load that, wondering if that would get around the need for quickjs-rs support for a WASM target. I haven't read much into wasmer myself.

l0calh05t commented 3 years ago

@xu-cheng, I wonder if it's possible to embed wasmer into this crate and use it to load a QuickJS wasm file. In wasmer readme's quickstart, it specifically shows loading a QuickJS wasm file and using it via the cli. If you can load that, wondering if that would get around the need for quickjs-rs support for a WASM target. I haven't read much into wasmer myself.

This seems like a very viable option, considering wasmers quickstart-example runs a pre-build quickjs

l0calh05t commented 3 years ago

The prebuilt quickjs module is a WASM version of the quickjs CLI, so not particularly useful. Would https://duktape.org/ be worth looking into? EDIT: I doubt duktape would work, seems to support only a tiny subset of ES6

l0calh05t commented 3 years ago

QuickJS on wasmer is definitely viable. Here's a proof of concept: https://github.com/l0calh05t/wasmer-katex-poc/ However, it doesn't fit the current Engine-API well

xu-cheng commented 3 years ago

Hi, sorry for the late response. Unfortunately, there are multiple issues with quickjs wasm.

The biggest issue is the wasm itself is not well maintained and not official project of quickjs. I also have some hesitation to distribute an unofficial (i.e., untrusted) binary file.
As you may already find out, the quickjs wasm is designed to be a CLI rather than a library. This wouldn't work well with this library, especially we want to only initiate katex js once.

I still think the best (maintainable) approach is to add existing well maintained backends like v8 and wasm-bindings. And if someone can maintain a rust version of quickjs (like slimjs), it would solve all our problems.

l0calh05t commented 3 years ago

The biggest issue is the wasm itself is not well maintained and not official project of quickjs. I also have some hesitation to distribute an unofficial (i.e., untrusted) binary file.

Agreed, the lack of maintenance isn't great. However, the source is available: https://github.com/saghul/wasi-lab/tree/master/qjs-wasi and while an untrusted binary isn't great, it is running in a sandboxed environment.

As you may already find out, the quickjs wasm is designed to be a CLI rather than a library. This wouldn't work well with this library, especially we want to only initiate katex js once.

Yes, that's what I meant when I wrote that it doesn't fit the Engine-API well (and is probably very inefficient)

I still think the best (maintainable) approach is to add existing well maintained backends like v8 and wasm-bindings. And if someone can maintain a rust version of quickjs (like slimjs), it would solve all our problems.

The problem with quickjs is the lack of Windows/MSVC support. There are forks with Windows/MSVC support, but the official version isn't viable for the main Rust on Windows platform. v8 is probably the only real option right now

Chaostheorie commented 3 years ago

Might also be worth looking into rust-mozjs, the rust binding to spidermonkey. These binding also include an example of using it with a string for evaluating as a script. Since spidermonkey is a relatively well tested JS engine and has support for most OS's, this might a viable alternative.

xu-cheng commented 3 years ago

I have now released katex 0.4.0. It now supports two new JS backends, i.e., Duktape and wasm-bindgen. If you want to support more JS backends, PR is welcome.

There are some unsolved issues.

The default feature selection is suboptimal. Currently, the default backend is QuickJS. To enable other backend, the crate users have to disable the default feature first. Ideally, the default feature should automatically select backend based on the platform. However, this is currently impossible due to the restriction of the Cargo.
The wasm-bindgen backend only supports browser for now. In Node.js, as JS codes are imported through eval, it reports Cannot find module error. We need to find a way to tweak the JS code to be correctly interpreted by Node.js. Help is needed. It has now been fixed by katex 0.4.1.

joseluis commented 2 years ago

Could deno be another possible candidate?

xu-cheng commented 2 years ago

@joseluis deno uses v8 as its js engine. PR is welcome to add v8 or other js engines support to this crate.