Deciding on a path forward with packaging

[tanneberger]

We have to decide on if we want to use nix as a backend or not.

How the Demo works

The demo reads the Barrel.toml then generates the nix code. Then the nix builder is called with evaluates the expression and builds the package. The actual target wrapper and lingua-franca packages are defined in lf-pkgs.

Barrel with Nix

If we decide to use nix as a backend installing libraries in your system won´t work anymore, because nix has a isolated build environment. So just make install in your system wont´t work. Libraries need to be packages or pulled into the environment, when nixpkgs didn´t already package it. Packaging something in nix is compared to other systems like debian trivial but Industry folks probably don´t have the nerves to do it. The easiest way to pull in other libraries is by pulling them in as a git-submodule (e.g. lf-alarm-clock)

Build Process

flowchart TB
    A[Start] -->B(Fetch Meta from packages)
    subgraph barrel
    B --> C(Generate Nix Code)
    end
    subgraph nix-env
    C --> D(Evaluate Nix)
    D --> E(Nix Deamon Builds Package)
    end

Nix Architecture

• the current lf-pkgs repo would stop holding package definitions and only define the different target wrappers.

• barrel would check which version is specified and then generate the correct fetchgitor fetchTarball

  The upside of using the nix-env would that we get a package definition and we can profit of the eco-system e.g. Remote-Builders, Package Caches ...

Barrel without Nix*

The difference between the two approaches is mainly that we have build everything ourself but don't have the constraints that are introduced by nix for reproduce-ability. This would include fetching, unpacking, moving stuff, invoking lfc.

flowchart TB
    A[Start] -->B(Fetch Meta from packages)
    subgraph barrel
    B --> C(manually fetch Tarballs)
    C -->  D(Move Sources and Setup Build-Env)
    D --> E(Invoke correctly parameterization lfc)
    end

Rust or Kotlin

[lhstrh]

As a recap of our discussion: it looks like Nix is very capable and could get us up an running quickly. We should, however, from the get go, offer an escape hatch for users that for whatever reason can't or don't want to use the Nix back end. If we design it like that from the start, we don't lock ourselves in too much. As for the implementation language, I think Rust is cool for a "clean slate" design, but I think there will significant value in sharing code between barrel and lfc (e.g., for parsing, etc.), which would be enabled by opting for Kotlin.

[lhstrh]

@oowekyala pointed out that if we do our job well, there shouldn't be any overlap between lfc and barrel, which I think is a good point. To me, this argument makes it more of a toss up between Kotlin and Rust...

[tanneberger]

Report

Suggested Design

• lfc should become a clean code generator that only produces source code
• barrel should start by invoking lfc and later develop into generating build instructions (cmake, etc. )
• similar case to cargo nobody invokes rustc manually every body does cargo b
• Discussing quirks about arduino with *.ino which is litterally C-Code for uploading software to the board
• Arduino compiles everything in src this needs some attention, because we can end up with multiple definitions.
• christian suggested that we should start with c, cpp and then move to more exotic targets.

[arengarajan99]

One update to make on the Arduino side that @petervdonovan brought up. We can get around lazy compilation issues with src folder by simply precompiling reactor-c as a static library file and use arduino-cli's library functionality to include reactor-c's static library directly without having to deal with a src folder. Essentially, the package manager would simply need to create a static .a library file from reactor-c, have arduino-cli install the library internally, then have our generated lfc code act as our sketch and include the library. This will require tinkering, but it will simplify the process greatly since we don't need to copy over files.

[cmnrd]

Cool! This is in line with the general approach that I would take with barrel. Barrel would:

1. Invoke lfc on all required .lf files to genrate target code
2. Build the runtime lib(s) for the specific platform and settings
3. Invoke the target compiler or build system to compile the generated code

[tanneberger]

nix is not used anymore