search

deislabs / spiderlightning

A set of WIT definitions and associated implementations to enable app developers to work at a faster pace and require less knowledge of the environment in which they are executing.
https://deislabs.github.io/spiderlightning
MIT License
308 stars 30 forks source link

set up containerd to run slight #97

Closed danbugs closed 2 years ago

KaiWalter commented 2 years ago

@danbugs what exactly would you like to achieve with this issue? Is it just containerizing one slight workload like in https://github.com/KaiWalter/spiderlightning/blob/hosting-service-fabric/cluster/cluster-test/Dockerfile)?

danbugs commented 2 years ago

Hey, @KaiWalter ~ Thank you for your question. I definitely should gone back and included descriptions to issues like this once we open sourced.

That said, this issue will most likely be a contribution to:

Are you familiar with Fermyon's Spin? If you check the containerd-wasm-shims repo, you'll see we've got a shim for it that helps serve a containerized Spin app. We hope to do something similar for Slight. I hope this clarifies the scope of the issue — let me know if it doesn't!

KaiWalter commented 2 years ago

Hi @danbugs, I see. That was actually, what I was talking about with @squillace 2 weeks ago: "get WASM/WASI workloads running in cluster or other environments without using OCI containers". For the moment I have to admit, that the repos you listed above are technically somewhat out of my league. So don't wait for me, please. Anyway as I want to use my vacation time for building up some Rust muscle I will definitely munch into those.

KaiWalter commented 2 years ago

@danbugs - After I figured out how runwasi is working, I can imagine to get slight and a module WebAssembly packaged up with a containerd-shim approach. But still using docker buildx to build the image. @squillace - But then why? Why go to all the length to patch WebAssembly execution into containerd ecosystem? Just to then make it easier to hook it up with K8S based hosting? Wouldn't just aiming for a clustered process orchestrator like Service Fabric be the simpler approach here? I did some preliminary tests and the only shortfall I currently see is, that SF up till now only supports Ubuntu up to 20.04 which makes patching some of the slight dependencies into the nodes somewhat hard. If we could get support for 22.04 nodes, that should be far more easier.

squillace commented 2 years ago

Hi @KaiWalter was out for a few days to repower. I have only partially repowered. :-| But YES, the objective will be to have a containerd-based integration into k8s. There are several reasons.

  1. customers who need a scale-out, non-proprietary control plane have by and large standardized on k8s. As a result, to reach both hyperscale and periphery compute environments k8s will be the primary early control plane that all can use in either proprietary services or connected oss.
  2. You should expect that early services, then, use the k8s host. But as understanding increases, other "schedulers" will be not only welcomed but sought out. Fermyon is touting Hashicorp's Nomad; in Azure, you can imagine several other orchestrators or schedulers; in oss, we know of several customers researching running real time oses that schedule wasm functions because they do not have container infra.
  3. containerd seems like it's for k8s but it really isn't: it's a generic process-starting system that can be integrated easily elsewhere. As example one, docker supports docker run --runtime=io.containerd.wasmtime.v1 now and more direct work will arrive.
squillace commented 2 years ago

Nonetheless, I also expect that standalone hosts like slight or spin or wasmcloud (which is its own orchestrator I note) will be popular in all areas in which k3s makes no sense as a control plane. As they should be.

KaiWalter commented 2 years ago

@squillace thanks for the explanation - got it. This is fascinating stuff. You know I've not been doing kind of "systems programming" since early 90ies. This really strikes a chord. I will be off the grid for a few days - also unwinding a bit. I'd really like to contribute. Do you perhaps run a more tactical forum like Discord, Gitter, ...? Also Teams/microsoft.com tenant would work for me.

squillace commented 2 years ago

Wait, @KaiWalter, you're on vacation but you want "tactical"? :-P

For work with slight, just post issues! everyone can see and learn, which is an advantage. For more focused issues, mail me at ralph.squillace@ .com and we'll set something up to make the vacation fun.

KaiWalter commented 2 years ago

@squillace I need this to unwind - https://twitter.com/ancientitguy/status/1560892996991782913 - with all those heavy duty enterprise projects I usually do, where technological challenges are easily trivialized by the complexities on organization, process or social level.

danbugs commented 2 years ago

~check out slight (previously, wasi-cloud) running on a PinePhone:

video link: https://user-images.githubusercontent.com/39843321/186492047-68cf490b-1629-4113-a6dc-bb8f01bc8304.mp4

P.S. might have to click on it to zoom in or view video

Mossaka commented 2 years ago

Closed this issue as we move containerd related issues to https://github.com/deislabs/containerd-wasm-shims