search

wasmCloud / wasmcloud-otp

wasmCloud host runtime that leverages Elixir/OTP and Rust to provide simple, secure, distributed application development using the actor model
Apache License 2.0
228 stars 48 forks source link

Create a distillery release #67

Closed autodidaddict closed 3 years ago

autodidaddict commented 3 years ago

Add distillery to the project and run through the process of manually creating a release. Automate this into the CI pipeline so that when a v0.x.y tag is created, a new distillery release is created for Linux, Mac, and Windows and attached to the release as a .tar.gz file, e.g. wasmcloud-host-x86_linux.tar.gz.

Let's also figure out a way to automate the generation of the aarch64 image.

brooksmtownsend commented 3 years ago

Finished items

Need to be done

Optional / Post MVP

@billyoung I'd be happy to pull up for a quick chat to go over the work that I've done, but the TL;DR is that the elixir runtime is packaged with a distillery release. This creates a standalone .tar.gz which can be unzipped and then ran using ./install_dir/wasmcloud_host foreground or ./install_dir/wasmcloud_host start. This, by itself, is an awesome system, but there are 2 parts that make it difficult.

  1. The elixir runtime system (ERTS) is bundled with the application by default. This is compiled for an operating system/architecture, so it has to match. We have the option to not include the elixir runtime, but if we do that then the user must install the exact same version of elixir and erlang that we built it with or the project will fail to run.
  2. We use Native Implemented Functions (NIFs) to bring in some of our Rust functionality as callable C functions. These must be compiled for an OS/ARCH, so even if we don't bundle the ERTS the NIFs will still have to match the host machine.

In order to tackle this, my first approach was to create a generic docker image where I used a different image for building the elixir runtime, rust NIFs, and for the distributed docker image I just copied and ran the application with a basic image (I used debian:buster-slim). This worked great for x86 linux, but started to fall apart for different architectures and OSes due to the complexity of cross compiling elixir and Rust, and no generally available docker images for MacOS. I still use this Dockerfile and the Dockerfile.release for the x86 linux build because it's efficient and creates our docker image for us, and I think it's generic enough that it could be used for aarch64 linux and maybe even Windows if you know how to set it up (I couldn't).

With the difficulty of macos and windows and cross compilation builds, I decided to go with using the github macos / windows runner for the next part. The current state of this effort is I have most of the macos build scaffolded out, just dealing with caching the erlang installation so we don't spend 30 minutes installing it every time. Once I finish that up, I'd love to handoff to you (Bill) to help drive this effort home!

autodidaddict commented 3 years ago

@brooksmtownsend is this issue ready to close?

brooksmtownsend commented 3 years ago

Closing in favor of #137 for clarity around what work has been done and what still needs to be implemented