Document how to use from Node.js/browser

[rrthomas]

I would like to use Topiary (compiled to WASM) as a library in a Node.js project.

This should clearly be possible, but it's not obvious how to separate Topiary from e.g. the demo web app.

Thanks for Topiary! I was easily able to add a formatter for a new language.

[yannham]

Hello,

I think you should be able to use the demo app as an example, as the playground is nothing special in the end. Bootstrap a simple React app (or whatnot). Then you can just import Topiary functions exported as WASM: https://github.com/tweag/topiary/blob/b4a842673b2849fd1efef23df39cc25e26a352c8/web-playground/src/App.tsx#L5-L9.

You still need to make this wasm-app stuff available somewhere. To do so, you can see that the CI just builds the crate topiary-playground - a simple wrapper around topiary-core to re-export the interesting bits to WASM - and copy the result to a new wasm-app directory in the playground's webapp: https://github.com/tweag/topiary/blob/b4a842673b2849fd1efef23df39cc25e26a352c8/.github/workflows/ci.yml#L48-L56.

The nix build .#topiary-playground command does some magic in addition to compiling the topiary-playground. If you really don't want or can't use Nix, you can also do it manually or using a script by looking at what the corresponding Nix part does: https://github.com/tweag/topiary/blob/b4a842673b2849fd1efef23df39cc25e26a352c8/default.nix#L139-L163.

There's no particular hidden coupling, to the best of my knowledge, between Topiary and the web playground, so it should indeed be reasonably straightforward to use Topiary from your own web app!

[rrthomas]

Thanks, I should have been more specific: my actual use-case is that I want to use Topiary as a library from a Node.js program. So you're quite right: I could probably work out how to use it in a web app from the example. But I felt that it would take a lot of effort to work out how to use it as a Node.js library. But I'll take a look at your explanation and see if I can transpose what you say from browser to Node.

[yannham]

I'm not well versed in the Node ecosystem, but then I think the important bit is that nix build .#topiary-playground should compile Topiary to WASM and pack it as a standard Node package (put under the local directory result), that you should then be able to use as any other NPM dependency (sometimes there are some gotchas with respect to loading WASM modules, but it depends on your precise setup)

[rrthomas]

Not expecting you to do anything @yannham, just noting I tried a non-nix build (as I'd like ideally to have this as part of a plain npm build) and had the following problem:

$ cargo build -p topiary-playground --no-default-features --target wasm32-unknown-unknown
  cargo:warning=/home/rrt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tree-sitter-0.22.6/src/./alloc.h:9:10: fatal error: 'stdio.h' file not found
  cargo:warning=    9 | #include <stdio.h>
  cargo:warning=      |          ^~~~~~~~~
  cargo:warning=1 error generated.

In other words, while building tree-sitter, the C compiler couldn't find stdio.h, which makes sense, as wasm32-unknown-unknown is a "bare-metal" target; but I have no idea what I'm supposed to do about it, or why it works in the nix build.

[yannham]

It's strange. From what I recall, topiary-playground shouldn't be using tree-sitter directly but rather tree-sitter-facade, which role is precisely to offer a reduced interface which also works when compiling to WASM. The Nix build doesn't seem to be doing much magic beside providing wasm-bindgen, binaryen and pkg-config in the PATH. Maybe Topiary devs would have some idea about that? @Xophmeister

[Xophmeister]

I'm afraid I'm not familiar with whatever magic Nix/Crane performs to build WASM targets. If you can get it to build with Nix, as @yannham suggested, at least that would allow you to move forward regarding Topiary and Node.js (which we're only presuming "just works" at the moment).

[rrthomas]

Thanks, both. Getting this working is not a high priority for me at present (I am content to invoke topiary via the command line from my Node program), but I will want to get it working at some point.