Coming up with a convention for using files in deployment and execution

[trkks]

In the tech-status-check today, using the WebAssembly component model that wasmtime supports was discussed as a major point of interest in order to support more reliable and high-level (i.e. non-pointer-based) communication between both 1) two separate supervisors and 2) the supervisor and WebAssembly module.

[trkks]

Might be better if files would just be files. By this I mean that they'd be read from the filesystem, which would then benefit from WebAssembly's capability-oriented file access.

With a concrete example of adapting the object inference module, which takes in two pointers (model and input image), we could wrap it into a single component that first reads the files into memory:

model file --\                /-- model_ptr+len --\
              +--> adapter --+                     +--> object inference
image file --/                \-- image_ptr+len --/

[trkks]

Ok. Now I'm just going with assuming, that if a module wants files to read or write, it will do so using WASI. Arguments and outputs to functions will be Wasm-primitives and interpreted as such (i.e. ditch the pointers).

OpenAPI descriptions will need to contain the files' names or paths relative to "module's root" for reading/writing. For example reading as input a file at image/to/manipulate.jpeg and sending as response file at manipulated/image.jpeg the RequestBody would have a content field such as (partly adapted from OpenAPI examples):

"schema": {
    "type": "object",
    "properties": {
        "file/to/manipulate.jpeg": {
            "type": "string",
            "contentMediaType": "image/png",
            "contentEncoding": "base64"
        }
    }
}

and the same is repeated in the (successful) ResponseObject's content field. The incoming request would then have to contain the field image/to/manipulate.jpeg and in return the outgoing response would have a field manipulated/image.jpeg.

The type being a file and not e.g. a primitive can be recognized from the contentMediaType.

This might be usable but in the end is total chewing gum.

[trkks]

...

"schema": {
    "type": "object",
    "properties": {
        "file/to/manipulate.jpeg": {
            "type": "string",
            "contentMediaType": "image/png",
            "contentEncoding": "base64"
        }
    }
}

...

Will add some "stage": "deployment" | "execution" field to the description as well, to indicate if the file is expected at deployment or execution time to prevent reloading e.g. a model file on every run.

[trkks]

Inference worked (not from deployment-time model file though), so merge and fix issues later when they come up (this branch has again been up long enough)