search

cartesi / rollups-node

Reference implementation of the Cartesi Rollups Node
Apache License 2.0
22 stars 64 forks source link

Investigate a new development language #72

Closed torives closed 1 year ago

torives commented 1 year ago

đź“š Context

Rust is a great language, with many desirable characteristics. However, its steep learning curve and the challenges it poses in terms of developer ergonomics have prompted us to explore alternative options. The two contenders are Go for its efficiency and concurrency model and Javascript/Typescript, as it would facilitate the node's integration with Sunodo and the possibility of a reader node running in the browser.

✔️ Solution

Create a small proof-of-concept project in each language to evaluate:

1) Go's integration with WASM and its limitations 2) How to adapt our current needs to Node.js/Javascript's concurrency model

These projects should provide us with enough information to make a decision.

torives commented 1 year ago

The original goal was to choose a new development language to replace Rust, evaluating Go and JavaScript mainly on the impact they would have on our (far away) goal of running a reader node in the browser. That might have been too ambitious, but we have learned some interesting things anyway.

Exploring the Go side of things, I searched for how to run Go code in a browser. I came upon three main possibilities:

  1. Compile Go to WASM with the default compiler This turned out to be a positive surprise, as it is quite easy and straightforward to do. Setting GOOS=js GOARCH=wasm was pretty much all that I needed to do to compile a simple HTTP server written in Go to WASM (goroutines and all). I was expecting it to fail as WASM doesn’t support sockets yet, but not only did the compilation succeed but it also didn’t throw any errors when executing the WASM binary in the browser. I have no idea why nothing blew up, but as expected I wasn’t able to connect to the server. I also wrote a very simple HTTP client to query http://catfact.ninja/fact, and everything worked smoothly.
  2. Compile Go to WASM with the TinyGo compiler TinyGo is a Go compiler for embedded systems that also supports a WASM target. It claims to produce smaller binaries but it doesn’t support the full standard library, as became evident when it failed to compile my HTTP server example. I didn’t explore any further.
  3. Compile Go to JS using GopherJS GopherJS compiles Go to pure JavaScript. Unfortunately, the project doesn’t have many contributors and seems to be lagging behind Go releases (the latest supported version is 1.18, the current version is 1.21) and general API support. I didn’t try it.

After some reading, testing, and thinking, I came to some conclusions:

  1. It’s not possible to take the node “as is” and run it inside a browser. A browser is a much more constrained environment and would require a redesign of the node instead of simply adapting the existing code.
  2. WASM modules are very limited in their capability to communicate with each other or with the OS, which reinforces the first point
  3. But on the other hand, WASM modules really shine as a way to perform complex computations efficiently in the browser. If we can modularize the native code in a way that tasks are abstracted behind a simple API that takes an input and spills out an output (like host-runner’s proof generation), then we have the opportunity to reuse this code inside the browser as a WASM module.
  4. And because of (1), the language used to build the native node has very little impact on how we go about building the reader node in the browser.
torives commented 1 year ago

After discussing these points within the Node Unit, we've concluded that Go is a suitable and in fact the preferred option between the two. Going forward, we'll start using it for new code.