containerd / runwasi

Facilitates running Wasm / WASI workloads managed by containerd
Apache License 2.0
1.01k stars 82 forks source link

Create an API for Pre-processing layers #504

Open jsturtevant opened 4 months ago

jsturtevant commented 4 months ago

We've added the ability to pre-compile layers in #405 and followed up with some improvements in #492.

During the iterations it was identified that a similar API could be used to do other pre-initialization work as well:

I would like to propose we create an API similar to the precompile but would replace it with something like process_layers or initialize_layers. This should also allow the shim to give us processed layers that are combined and the id's of ones the want to remain untouched.

One possible way to do this would look something like the precompile layer (open to suggestions):

struct ProcessedLayers {
  layers: Vec<Layer>
  unique_cache_id: Option<String> // return None if nothing to do
}

enum Layer {
 Original(digest)  // for files we want untouched
 Processed(Vec<u8>)
}

fn precompile(&self, _layers: &[WasmLayer]) -> Result<ProcessedLayers> {
        // do work
}

On the shim side we could take the layers create a new file that stores these digests and link it to the original image. Upon reloading the image we could look up the file, load those digests.

Questions: