A minimal packaging/dependency proposal

[joshgoebel]

I'm putting this idea forward here as one idea for a way to consider for simple dependencies (vs vendoring everything). Hopefully community discussion can help evolve the idea or lead to even better ideas.

What I tried to solve:

• An easy way to codify a list of dependencies and their versions
• An easy/repeatable way to install those dependencies into a project's wren_modules
• A way to do this without introducing too many further dependencies (bash, Python, etc)... which meant it had to be as much pure Wren as possible.

I just needed the bare minimum that would allow students to install Wren dependencies needed to run Wren tests for Exercism. I started out vendoring libraries but that was proving quite problematic and frustrating to maintain.

What I didn't try to solve:

• Dependencies that have further dependencies. (though these could all be specified and commented in a single project level package.wren)
• Providing any type of separate executable other than the package.wren itself...
• Dynamically updating the package.wren file via some sort of utility... right now it's all hand maintained.
• Upgrading dependencies. (though wipe/reinstall isn't hard)
• Network access of any type (it just uses git to fetch needed dependencies - which must be git repos)

Obviously one could imagine a super complex system here akin to npm/yarn, but I was going for something much simpler... it's currently less than 100 total lines of Wren code.

---

The idea, a package.wren file to express simple first-tier dependencies of a package. This file is of course pure Wren. One could imagine additional getters (or perhaps attributes?) to express details of the package.

import "wren-package/package" for WrenPackage, Dependency

class Package is WrenPackage {
  construct new() {}
  name { "wren-test-runner" }
  dependencies {
    return [
      Depencency.new("wren-testie", "0.1.0", "https://github.com/joshgoebel/wren-testie.git")
    ]
  }
}

// this allows for installation, query, etc, just by running the package file
Package.new().default()

Then you simply run the package ./package.wren to get information about it...

Usage:
./package.wren install

wren-test-runner dependencies:
- wren-testie 0.1.0

Or ask it to install itself:

 - installing wren-testie 0.1.0
 - [R] git clone -q https://github.com/joshgoebel/wren-testie.git wren_modules/wren-testie
 - [R] git checkout --detach 0.1.0
HEAD is now at 921b912 show variety
 * 1 dependency(s). All good.

---

This idea has already been implemented via wren-package and is in use in the real world of Exercism. It currently works with wren-console. Right now the big missing wren-cli pieces I see preventing something like this are:

• Process.exit #74
• Process.exec #94
• Bug with directory creation #115
• node_modules can only resolve to a single directory making a "global" + local installation of packages impossible (this quite limits the usefulness I think) Related: #78

It seems the most natural thing would be to install a packager globally and then use it locally with projects... ie the packager might live in $HOME/wren_modules and then your project's dependencies in $HOME/projects/sample/wren_modules. It seems very natural that the package manager itself needs to be installed globally, outside the scope of your project... I guess I'm also very used to how the JavaScript ecosystem (and others) do this. I actually set things up this way without even thinking about it then was bummed when it failed - and had to go add support for resolving imports from multiple node_modules directories...

One could also imagine just compiling the package management code into the CLI as well, alleviating the need for it to be installed at all - though this makes it a lot more tied to the CLI installation.

Anyways, just food for thought.

[Orcolom]

I have some quick ideas:

(I haven't checked if the syntax compiles)

// --- Option B
var Package = WrenPackage.new("wren-test-runner", "1.0.0", [
    Depencency.new("wren-testie", "0.1.0", "https://github.com/joshgoebel/wren-testie.git")
])

// --- Option C
var Package = PackageBuilder.new("wren-test-runner")
Package.setVersion("1.0.0")
Package.addDependency("wren-testie", "0.1.0", "https://github.com/joshgoebel/wren-testie.git")
Package.lock() // stop edits

// --- Option D
var Package = {
    "name": "wren-test-runner",
    "version": "1.0.0"
    "Depencency": [
        {
            "name": "wren-testie",
            "version": "0.1.0",
            "url": "https://github.com/joshgoebel/wren-testie.git"
        }
    ]
}

• A, C, D are more future proof
• Don't like D as its all strings, but json users would easily understand
• B is limited by its constructor
• C is as flexible as creating a new class without the need to create a new class

[clsource]

I personally like the way https://github.com/apple/swift-package-manager works

Example

https://raw.githubusercontent.com/vapor/leaf/main/Package.swift

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "leaf",
    platforms: [
       .macOS(.v10_15)
    ],
    products: [
        .library(name: "Leaf", targets: ["Leaf"]),
    ],
    dependencies: [
        .package(url: "https://github.com/vapor/leaf-kit.git", from: "1.0.0"),
        .package(url: "https://github.com/vapor/vapor.git", from: "4.0.0"),
    ],
    targets: [
        .target(name: "Leaf", dependencies: [
            .product(name: "LeafKit", package: "leaf-kit"),
            .product(name: "Vapor", package: "vapor"),
        ]),
        .testTarget(name: "LeafTests", dependencies: [
            .target(name: "Leaf"),
            .product(name: "XCTVapor", package: "vapor"),
        ]),
    ]
)

This can be translated to Package.wren like this

// wren-cli-version:0.5
import "package" for Package as WP

var Package = WP.new()
              .name("leaf")
              .platforms([
                  WP.os().macOS(WP.macOS.v10_15)
              ])
              .products([
                  WP.library().name("Leaf").targets(["Leaf"])
              ])
              .dependencies([
                  WP.package().url("https://github.com/vapor/leaf-kit.git").from("1.0.0"),
                  WP.package().url("https://github.com/vapor/vapor.git").from("4.0.0")
              ])
              .targets([
                  WP.target().name("Leaf").dependencies([
                      WP.product().name("LeafKit").package("leaf-kit"),
                      WP.product().name("Vapor").package("vapor")
                  ]),
                  WP.testTarget().name("LeafTests").dependencies([
                      WP.target().name("Leaf"),
                      WP.product().name("XCTVapor").package("vapor")
                  ])
              ])