lpil commented 5 years ago

It would be fun and also useful if we could compile Gleam to native code. This comes up frequently in our discord chats, so let's keep some notes here for future reference.

References

Codegen

https://leonardschuetz.ch/blog/nan-boxing/

https://piotrduperas.com/posts/nan-boxing (with benchmark)

https://bernsteinbear.com/blog/compiling-a-lisp-2/

WASM

https://github.com/WebAssembly/binaryen

https://kripken.github.io/blog/wasm/2019/07/16/asyncify.html (used by TinyGo)

Memory layout

rockwotj commented 2 years ago

Any interest in directly compiling to WASM?

lpil commented 2 years ago

edit: I have changed my mind! wasm is good target.

dch commented 2 years ago

Perhaps it’s possible to have an erts single image binary and slap a gzip of beam files on the end of it?

rockwotj commented 2 years ago

Warp has an Erlang example: https://github.com/dgiagio/warp/blob/master/examples/erlang/README.md

lpil commented 2 years ago

Perhaps it’s possible to have an erts single image binary and slap a gzip of beam files on the end of it?

@dch I would like that but in the far future I'd also like a native Gleam that could be used when more single threaded performance or minimal memory usage is required. It could integrate with Erlang based Gleam using NIFs. One for the far future :)

dch commented 2 years ago

@rockwotj that's neat but basically a self-extracting archive with some smart caching. I'd like to have erts itself be able to handle this case. Not really a topic for here, though!

dch commented 2 years ago

@lpil oh yeah I see what you meant now. I'd love fast-executing command line tools, for example. No need for NIFs initially.

rockwotj commented 1 year ago

For what it's worth I've been (slowly) hacking on a C++ target output for Gleam.

I chose C++ over C because templates, (and lamdbas) are really nice to enforce typesafety and not mess with a bunch of void pointers that plain C would need.

So far math expressions, functions, records (with hierarchy and shared field support) and reference counting via std::shared_ptr are supported. There is certainly a lot of work to go, but I hope to slowly chip away at progress. There are a couple of open questions around things like external types and how the higher level features above the package-compiler will work with complexity of invoking C++ compilers (let alone supporting all the platform specific stuff and things like dependency management).

Initially I really wanted to output golang, as it has a lot of nice parallels with erlang (goroutines, GC, error handling is "result based", etc), but Go has a very different module system than Gleam and generics are lacking in some aspects.

Anyways I am happy to discuss further and I'll make sure to post if I ever "catch up" with all the features in the language to discuss upstreaming some of this work if there is interest.

lpil commented 1 year ago

That's very cool! C++ would probably be what I go for if I were to make a native backend today. Looking forward to seeing what you make :)

I feel there likely is a shared IR that both JS and C++ could use. A simple proceedural language which typed Gleam could be lowered to. Possibly not practical right now, but perhaps in future.

inoas commented 1 year ago

A shared IR would mean we could write arbitrary targets against for that IR? Say one on JVM/Java?

Edit: If FFI could also target that IR language, could we then abstract away FFI for common libs so that they run on all targets supporting the IR?

lpil commented 1 year ago

A shared IR would mean we could write arbitrary targets against for that IR? Say one on JVM/Java?

It would reduce the amount of work to add or maintain procedural language backends, aye.

If FFI could also target that IR language, could we then abstract away FFI for common libs so that they run on all targets supporting the IR?

I think you'd still need code per platform as they'd all have different APIs and characteristics

rockwotj commented 1 year ago

Firefly might be interesting for gleam. You could support WASM by leveraging the existing Erlang target and not have to change semantics. If you wanted to be able to then run them without a wasmengine it seems like wasmer can do that.

lpil commented 1 year ago

Yup! We've spoke in the past about Gleam exposing type information to Firefly so we can enable LLVM type based optimisations too.

gertvv commented 1 month ago

I've been toying a bit with WASM-GC and transcribed some Gleam code into it, taking some inspiration from the JavaScript backend and stdlib implementation: https://gist.github.com/gertvv/39475c39f6419b5a31786e4f64231da6

I'd be interested in attempting to compile a similar subset of Gleam to WASM-GC. Any pointers for where to start on such a thing?

rockwotj commented 1 month ago

A long time ago when Gleam was not stable, I was creating a C++ backend. The first thing was to create an IR to lower gleam to something more procedural: https://github.com/rockwotj/native-gleam/blob/main/compiler-core/src/ir.rs

Although Wasm has some control flow that is a little more functional like so maybe that isn't helpful here.

I would strongly consider using binaryen in the backend layer, it will enable optimizations and you'll have an easier time generating code using it: https://github.com/WebAssembly/binaryen/wiki/Compiling-to-WebAssembly-with-Binaryen#c-api

I didn't see any rust bindings so that will be needed to be generated.

lpil commented 1 month ago

There is actually a prototype wasm compiler that uses wasm GC. It was made by Danielle for a university project.

gertvv commented 1 month ago

Is that prototype somewhere public? I haven't come across it.

Anyway I've just found out glance exists so I might toy with a Gleam implementation to start.

lpil commented 1 month ago

Here it is: https://github.com/DanielleMaywood/fyp-gleam

timjs commented 2 weeks ago

One of my bachelor students is working on this as well. I'll point him to @DanielleMaywood's work.

Do you already have a thesis about this Danielle? Or some docs about what's working and what not? Could be helpful for @Harmful-Alchemist.

DanielleMaywood commented 2 weeks ago

One of my bachelor students is working on this as well. I'll point him to @DanielleMaywood's work.

Do you already have a thesis about this Danielle? Or some docs about what's working and what not? Could be helpful for @Harmful-Alchemist.

My dissertation is available here. I don't have any documentation for what is missing, but the main two features that are missing are FFI and support for BitArrays.

SichangHe commented 2 weeks ago

Firefly might be interesting for gleam. You could support WASM by leveraging the existing Erlang target and not have to change semantics. If you wanted to be able to then run them without a wasmengine it seems like wasmer can do that.

Firefly is unfortunately archived as of recently.

Harmful-Alchemist commented 2 weeks ago

One of my bachelor students is working on this as well. I'll point him to @DanielleMaywood's work. Do you already have a thesis about this Danielle? Or some docs about what's working and what not? Could be helpful for @Harmful-Alchemist.

My dissertation is available here. I don't have any documentation for what is missing, but the main two features that are missing are FFI and support for BitArrays.

Very interesting, thank you!

SichangHe commented 2 weeks ago

Wasmtime GC implementation tracking issue: https://github.com/bytecodealliance/wasmtime/issues/5032

Once Wasmtime implements GC, Lunatic is probably going to get it, making it a very interesting Erlang-like target.

SichangHe commented 2 weeks ago

Maybe the other good approach would be to compile to Dart, which has the Dynamic type and also compiles to native code.

gleam-lang / gleam

Native or wasm compiler backend for Gleam #109

References

Codegen

WASM

Memory layout

Wren

OCaml

Grain

Scheme

Skybison