Feature: Static file serving

[technosophos]

The goal of this feature is to take some files from a bindle and serve them via a Wasm module in the bindle. The target should be a wasm module that handles static file serving in a predictable and configurable way.

General overview:

• One bindle can have more than one Wasm module
• One module may serve out static assets
  • Ideally, we would be able to write a pretty good static file serving module and re-use it all over the place
• Assets can be clearly identified as static assets that should be served at an understood path.

A Bindle with Static Assets and Multiple Modules

Here is an example invoice.toml with two modules and some files that should be served statically, along with another file that should never be served statically. I have omitted a bunch of boilerplate for brevity.

bindleVersion = "1.0.0"

# The top-level data
[bindle]
name = "example.com/static"
version = "1.0.0"

[[group]]
name = "static-files"
satisfiedBy = "allOf"

[[group]]
name = "private-files"
satisfiedBy = "allOf"

# An example of a custom app
[[parcel]]
label.name = "myapp.wasm"
label.mediaType = "application/wasm"
conditions.requires = ["private-files"]
features.wagi.path = "/app"   # From the hippofacts

# An example module that just serves static files
[[parcel]]
label.name = "static.wasm"
label.mediaType = "application/wasm"
conditions.requires = ["static-files"]
features.wagi.path = "/static/..."  # Note the wildcard /..., which matches all subpaths

# These three parcels are static files to be served by static.wasm
[[parcel]]
label.name = "img/penguin.jpg"
label.mediaType = "image/jpeg"
conditions.memberOf = ["static-files"]
features.wagi.file = true   # Generated from hippofacts, designates this as a member of the `files = []` array in hippofacts

[[parcel]]
label.name = "img/cassowary.jpg"
label.mediaType = "image/jpeg"
conditions.memberOf = ["static-files"]
features.wagi.file = true

[[parcel]]
label.name = "css/style.css"
label.mediaType = "text/css"
conditions.memberOf = ["static-files"]
features.wagi.file = true

# This asset is private, is mounted to myapp.wasm, and can't be served via static.wasm
[[parcel]]
label.name = "super/secret/data.txt"
label.mediaType = "text/plain"
conditions.memberOf = ["private-files"]
features.wagi.file = true

(See comments on https://github.com/deislabs/hippofactory/pull/15 for how this would be generated from hippofacts)

Serving the Files

The bindle above is mounted to some URI paths:

• http://example.com/app/ is mapped to myapp.wasm
• http://example.com/static/ and all subpaths are sent to static.wasm

When WAGI encounters the features.wagi.file = true flag, the parcel is considered a file and is mounted as a file, attached to any modules that require this parcel.

In the bindle above, the mayapp.wasm module would have one file available to it, effectively mounted to the path /super/secret/data.txt. It would be able to access that data however it saw fit.

The static.wasm module has the following paths available, mounted by Wagi:

• /img/cassowary.jpg
• /img/penguin.jpg
• /css/style.css

The static.wasm module effectively can translate HTTP paths to static file paths. Here's basic pseudocode:

var path_info = env.PATH_INFO            // Example: /static/img/penguin.jpg
var prefix = env.X_MATCHED_ROUTE  // Example: /static

var file = path_info.trim_prefix(prefix)  // Gives us /img/penguin.jpg
var mime = guess_mime_by_extension(file) // gives us image/jpeg

// Obviously, we'd want to do something more memory-efficient
var data = fs.read_file(file)

// Print a content-type header, an empty line, then the data
stdout.println("Content-Type: " + mime)
stdout.println("")
stdout.print(data)

[itowlson]

So this is an encoding of the WAGI modules.toml inside a Bindle invoice.toml? Sorry if I am missing something, but this raises more questions for me about the relationship between WAGI and Bindle that I can't figure out from the specs - please do point me if this is already documented, or is this part of the proposal/exploration?

Regarding static files, the proposal seems fine as far as Bindle goes, though the requirement for the user to package static.wasm as part of every bindle with assets could be onerous. It took me a while to grok what you were trying to say about private files, but I think what you are getting at is that fundamentally all file mounts are private; it's up to a module's code which, if any, it chooses to serve, and this is not expressed in the invoice, only in the code. static.wasm happens to serve all files mounted into it, but the secret file isn't mounted there because the secret file isn't part of any group that static.wasm requires. I'd suggest dropping the comment and instead having a paragraph or table stating which files are mounted where.

Are we going to support static files via modules.toml too, or do static files require Bindle?

[itowlson]

Oh, or would you just use the volumes directive in modules.toml (and map static.wasm as the handler)?

[itowlson]

The current implementation does not recognise module parcels with conditions:

https://github.com/deislabs/wagi/blob/4b237ac12f34404d428ab15e6419559ea4932499/src/runtime/bindle.rs#L43

We'll need to review that if we want to use the requires condition to bring in static files.

(Also, this is a problem for hippofactory, which always emits a group and requires condition even if the group is empty. I can fix that on the hippofactory side though, if there is a reason we need to keep the filter this way.)

[technosophos]

Closed by https://github.com/deislabs/wagi-fileserver