wasm-pkg-tools

Tools to package up Wasm Components

This repo contains several Rust crates that can be used for fetching and publishing Wasm Components to OCI or Warg registries. It is also the home of the wkg command line tool, which exposes all the functionality of the libraries. The first (but not only) focus of this project is allow for fetching of Wit Interfaces stored as components for use in creating components. It can also be used to fetch and publish component "libraries" to/from a registry.

Installation

Right now installation of wkg is manual. You can either clone the repo and build from source, or download a pre-built binary from the releases page and add it to your PATH. In the future we will add this tool to various package managers. If this is something you'd like to help with, please feel free to open some PRs!

If you have a Rust toolchain installed, you can also install wkg with one of the following options:

cargo install wkg

If you have cargo-binstall installed, you can also install wkg with a pre-built binary:

cargo binstall wkg

Configuration

The wkg tool and libraries use a configuration file to store settings. This config file is still subject to change but we will try to keep it backwards compatible as we continue to develop the tool. This config file is meant to be used by both wkg and also any other language specific component tooling that wants to fetch from registries. This should allow for a single configuration file that can be used by all tooling, whether that be wkg or some other tool that isn't written in Rust.

The default location is $XDG_CONFIG_HOME/wasm-pkg/config.toml on unix-like systems and {FOLDERID_RoamingAppData}\wasm-pkg\config.toml on Windows but this can be overridden with the --config flag. Examples of this are found below:

The configuration file is TOML and currently must be edited manually. A future release will include an interactive CLI for editing the configuration.

Below is an annotated example of a configuration file that shows all the available options.

# The default registry to use when none is specified. Generally this is wasi.dev, but can be set
# for cases when a company wants to use a private/internal registry.
default_registry = "acme.registry.com"

# This section contains a mapping of namespace prefixes (i.e. the "wasi" part of "wasi:http") to
# registries. This is used to determine which registry to use when fetching or publishing a
# component. If a namespace is not listed here, the default registry will be used.
[namespace_registries]
wasi = "wasi.dev"
example = "example.com"

# This overrides the default registry for a specific package. This is useful for cases where a 
# package is published to multiple registries. 
[package_registry_overrides]
"example:foo" = "example.com"

# This section contains a mapping of registries to their configuration. There are currently 3
# supported types of registries: "oci", "warg", and "local". The "oci" type is the default. The
# example below shows a use case that isn't yet super common (registries that speak multiple protocols)
# but is included for completeness.
[registry."acme.registry.com"]
# This field is only required if more that one protocol is supported. It indicates which protocol
# to use by default. If this is not set, then the fallback (oci) will be used.
default = "warg"
[registry."acme.registry.com".warg]
# A path to a valid warg config file. If this is not set, the `wkg` CLI (but not the libraries) 
# will attempt to load the config from the default location(s).
config_file = "/a/path"
# An optional authentication token to use when authenticating with a registry.
auth_token = "an-auth-token"
# An optional key for signing the component. Ideally, you should just let warg use the keychain
# or programmatically set this key in the config without writing to disk. This offers an escape
# hatch for when you need to use a key that isn't in the keychain.
signing_key = "ecdsa-p256:2CV1EpLaSYEn4In4OAEDAj5O4Hzu8AFAxgHXuG310Ew="
[registry."acme.registry.com".oci]
# The auth field can either be a username/password pair, or a base64 encoded `username:password` 
# string. If no auth is set, the `wkg` CLI (but not the libraries) will also attempt to load the
# credentials from the docker config.json. This field is also optional and if not set, anonymous
# auth will be used. If you're just pulling from a public registry, this is likely not required.
# If you're using a private registry and/or publishing, you'll almost certainly need to set this.
auth = { username = "open", password = "sesame" }
# This is an optional field that tells the OCI client to use a specific http protocol. If this is
# not set or not one of the accepted values of "http" or "https", then the default (https) will
# be used.
protocol = "https"
[registry."acme.registry.com".local]
# This is a required field that specifies the root directory on a filesystem where the components
# are stored. This is mostly used for local development and testing.
root = "/a/path"

# If a registry only has a config section for one protocol, then that protocol is automatically
# the default. The following is equivalent to:
# [registry."example.com"]
# default = "warg"
# [registry."example.com".warg]
# config_file = "/a/path"
[registry."example.com".warg]
config_file = "/a/path"

Well-known metadata

The wkg tool and libraries expect a registry.json file to be present at a specific location to indicate to the tooling where the components are stored. If a registry was example.com, then the tooling will attempt to find a registry.json file at https://example.com/.well-known/wasm-pkg/registry.json.

A full example of what this registry.json file should look like is below:

{
  "preferredProtocol":"warg",
  "warg": {"url":"https://warg.example.com"},
  "oci": {"registry": "ghcr.io", "namespacePrefix": "webassembly/"}
}

The preferredProtocol field is optional and specifies which protocol the registry expects you to use in the case where it supports both OCI and Warg. If both warg and oci config is in the registry.json it is highly recommended that this field be set.

For the oci config, the registry field is the base URL of the OCI registry, and the namespacePrefix field is the prefix that is used to store components in the registry. So in the example above (which is for wasi.dev), the components will be available at ghcr.io/webassembly/$NAMESPACE/$PACKAGE:$VERSION (e.g. ghcr.io/webassembly/wasi/http:0.2.1).

For the warg config, the url field is the base URL of the Warg registry used when connecting the client. Namespacing for warg is built in to the protocol.

Please note that for backwards compatibility, with previous tooling and versions of the wkg tool, you may also encounter a registry.json file that looks different. These files are still supported, but should be considered deprecated.

For OCI registries, the JSON looks like this:

{
        "ociRegistry": "ghcr.io",
        "ociNamespacePrefix": "webassembly/"
}

For Warg registries, the JSON looks like this:

{
  "wargUrl": "https://warg.wa.dev"
}

Conventions for storing components in OCI

Astute observers will note that OCI requires a specific structure for how those components are stored. To be clear, this does not apply to deployable artifacts (such as those used by various runtimes), but only to WIT components or library components. Based on the information in the registry.json file, the base URL and namespace prefix will be joined together with the namespace and package name to form the full URL. So if you have a custom company namespace called acme. Then a package called acme:foo should be stored with the name acme/foo. If we use the registry.json file from the example above, then the component will be stored at ghcr.io/webassembly/acme/foo:0.1.0. Please note that the tag MUST be a valid semantic version or the tooling will ignore it when pulling.

Contributing

Want to join us? Check out our "Contributing" guide and take a look at some of these issues:

• Issues labeled "good first issue"
• Issues labeled "help wanted"

License

^{Licensed under Apache-2.0 WITH LLVM-exception}

_{Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license with LLVM-exception, shall be licensed as above, without any additional terms or conditions.}