search

cncf / sandbox

Applications for Sandbox go here! ⏳📦🧪
Apache License 2.0
136 stars 22 forks source link

[Sandbox] SpinKube #90

Open endocrimes opened 8 months ago

endocrimes commented 8 months ago

Application contact emails

danielle@fermyon.com, c.voigt@reply.de, fcastelli@suse.com, jiazho@microsoft.com

Project Summary

SpinKube is an open source platform for efficiently running (containerless) Spin-based WebAssembly (Wasm) applications on Kubernetes.

Project Description

SpinKube is a Kubernetes-based platform for running serverless Wasm without a container. It achieves this by combining a few technologies: A containerd-shim-spin that can directly run Spin applications from OCI Artifacts without a Container, A runtime-class-manager (formerly KWasm) operator that manages the lifecycle of WebAssembly runtimes on Kubernetes Hosts, and the Spin Operator which manages the configuration and lifecycle of Spin applications.

This combination provides excellent developer and operator experience when running Wasm on Kubernetes.

SpinKube provides many benefits over container-based serverless compute:

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/spinkube

Project repo URL in scope of application

N/A

Additional repos in scope of the application

No response

Website URL

https://spinkube.dev

Roadmap

https://github.com/orgs/spinkube/projects/5/views/7

Roadmap context

The SpinKube Project aims to integrate WebAssembly applications seamlessly into the Kubernetes and the broader CNCF Landscape. To achieve those goals there are up-and-downstream enhancements to make across Observability, Kubernetes Node management, and scaling systems. Although not yet fully realized in the GitHub Project, the roadmap includes bringing automatic OpenTelemetry instrumentation to Wasm applications and having runtime-class-manager handle the lifecycle of installing and upgrading complex runtimes on Kubernetes nodes.

We believe in the future of WebAssembly as the next wave of efficient Serverless compute.

Contributing Guide

https://www.spinkube.dev/docs/spin-operator/contributing

Code of Conduct (CoC)

https://github.com/spinkube/governance/blob/main/CODE_OF_CONDUCT.md

Adopters

No response

Contributing or Sponsoring Org

Fermyon Technologies, Microsoft, Liquid Reply, SUSE

Maintainers file

https://github.com/spinkube/governance/issues/13

IP Policy

Trademark and accounts

Why CNCF?

The CNCF hosts a vibrant and diverse community of developers and organizations. The CNCF is able to provide a shared governance model and neutral home for the project, allowing for collaboration among various vendors and end-users, fostering the creation of a solution that delivers collective benefits to all stakeholders within the ecosystem.

Benefit to the Landscape

To expand the range of serverless platform options in the CNCF ecosystem and bring the benefits of Wasm to the Kubernetes community. Many end-users are looking for options that allow them to give their developers a serverless solution that is cost-efficient while offering excellent performance. SpinKube offers a seamless way to do this by leveraging the power of Wasm and deeply integrating it into various layers of Kubernetes for integration with the whole ecosystem.

Cloud Native 'Fit'

Landscape: Serverless

TAGs: TAG Runtime + TAG App Delivery

TAG Runtime is a natural fit for much of SpinKube, especially as the home of WG-WASM. We already integrate deeply with ContainerD through runwasi, and aim to work alongside other common runtimes too. These discussions present an opportunity to enhance the Kubernetes ecosystem, especially for lightweight environments with WebAssembly runtimes.

TAG App Delivery is also relevant to SpinKube - Our primary distribution mechanism for Wasm artifacts are OCI Artifacts, which ties into the work of wg-artifacts.

Cloud Native 'Integration'

SpinKube unites Wasm applications and Kubernetes in a way that is natural to developers who use Kubernetes today - by running applications as deployments and creating regular Kubernetes Service’s. This means you can benefit from your existing Service Meshes, O11y Pipeline’s, CI+CD, and scaling solutions while also benefiting from the portability and efficiency of WebAssembly.

SpinKube is also built on the shoulders of giants - including OCI Artifacts, runwasi in the containerd project, Kubebuilder, and OpenTelemetry.

Cloud Native Overlap

The WasmEdge, wasmCloud, knative, and krustlet projects in the CNCF ecosystem have some overlapping objectives with SpinKube. The SpinKube Project aims to integrate WebAssembly applications seamlessly into the Kubernetes and the broader CNCF Landscape.

The CNCF WasmEdge, SpinKube, and CNCF wasmCloud each use a different Wasm runtime, wasmEdge, Spin, and wasmCloud, respectively. While each of them have a differing developer experience for building applications, they each have some level of support for running their applications on Kubernetes.

According to the wasmCloud documentation, "wasmCloud is compatible with, but not dependent on, Kubernetes". With wasmCloud, Pods in a cluster are the wasmCloud "hosts" and contain N Wasm applications. On the other hand, SpinKube was designed and built for running WebAssembly on Kubernetes - and there is a 1:1 relationship between Pods and Wasm applications. This is due to SpinKube leveraging the containerd/runwasi project to natively execute Wasm applications. Since all Wasm apps are represented as Pods, you can easily use existing tooling such as Cilium, Istio, HPA, KEDA, and Dapr with SpinKube.

WasmEdge is a runtime for embedded functions, microservices, udf, smart contracts, and IoT devices. The WasmEdge Runtime is excellent for packaging inside your software, but is more akin to WasmTime (the runtime used by Spin) than it is to Spin itself. Spin is hyper-focused on event driven serverless applications (that respond to events from HTTP requests/Queue messages/...), and SpinKube relies on that developer experience and programming model as we continue to integrate into the Cloud Native Ecosystem. Both projects leverage runwasi to run on Kubernetes, and as such SpinKube’s runtime-class-manager not only simplifies configuring the Spin shim but can also manage the WasmEdge shim if that runtime better suits your needs, or you would like to use both.

SpinKube fits in a similar domain as serverless projects in the Cloud Native ecosystem. Like Knative, SpinKube is targeted at engineers who build serverless platforms and want a Kubernetes native experience. The defining distinction is that SpinKube applications are Wasm applications, giving them a greater level of isolation (as every request is fully isolated), fast start-up times, and simplified management of system dependencies.

Krustlet is another project in the CNCF space that runs Wasm on Kubernetes; however, it is no longer maintained - the team that developed it did go on to develop runwasi however.

Similar projects

Knative, wasmCloud, wasmEdge, krustlet

Landscape

Not yet

Business Product or Service to Project separation

N/A.

Project presentations

SpinKube Tutorial - https://www.youtube.com/watch?v=TTXAUSicj04

Project champions

No response

Additional information

No response

devigned commented 8 months ago

I am a big +1 to this proposal. However, I believe it is significant that the namesake project at the core of SpinKube, Spin, is itself not a CNCF project. My main concern regarding this is that if there was a change of licensing for Spin, then the SpinKube collection of projects could be at risk.

boomskats commented 8 months ago

This proposal is really interesting (the demos at kubecon were great), but I do find some of the statements above confusing and potentially misleading.

This opening statement describing the cloud native overlap:

The CNCF WasmEdge, SpinKube, and WasmCloud each use a different Wasm runtime, wasmEdge, Spin, and wasmCloud, respectively.

explicitly clashes with this statement in the paragraph immediately following it:

WasmEdge [...] is more akin to WasmTime (the runtime used by Spin) than it is to Spin itself.

Is Spin itself a runtime that SpinKube depends on (as suggested by the first statement), or does it use Wasmtime (as suggested by the second)?

As far as I'm aware neither Spin or wasmCloud are runtimes. According to Spin's own readme, it is a framework which uses Wasmtime, and according to this blog post, wasmCloud also uses Wasmtime (it does not use 'wasmCloud'). If both wasmCloud and Spin use Wasmtime for their runtimes, what is the purpose of that opening statement?

I'm also not sure why this text refers to WasmEdge as 'CNCF WasmEdge' and wasmCloud as simply wasmCloud, given that they are both CNCF sandbox projects and I've generally seen wasmCloud referred to CNCF wasmCloud elsewhere.

Lastly, Taylor Thomas, one of the original developers of Krustlet that the above text refers to, recently wrote a blog post on this subject which I feel is a relevant read.

endocrimes commented 8 months ago

I'm also not sure why this text refers to WasmEdge as 'CNCF WasmEdge' and wasmCloud as simply wasmCloud, given that they are both CNCF sandbox projects and I've generally seen wasmCloud referred to CNCF wasmCloud elsewhere.

@boomskats that is entirely unintentional - in the place we do that I'll update it for wasmCloud too

Is Spin itself a runtime that SpinKube depends on (as suggested by the first statement), or does it use Wasmtime (as suggested by the second)?

WRT "runtime" - I'm going to try and do some wordsmithing to find a better word, but originally they (Spin and wasmCloud) were referred to as "runtimes" because from an end-user perspective they somewhat are in this context. I'm on holiday until Tuesday so that'll take a bit longer to change.

kapilt commented 7 months ago

It seems a little odd to have an operator and accompanying in a foundation for a base project not in a foundation which is also reliant on a venture backed startup. Ie. It’s unclear why spin is not part of this. Given the zeitgeist and heightened sensitivity to companies changing project licenses to non oss, it seems like a non starter unless the project is also in a foundation. else cncf as a community runs the risk of being foundation laundered on accessories for a core that could end up in the same place, re non oss.

Re other foundation homes for spin, Byte code alliance projects are typically lower level runtime/spec oriented versus orchestration here. Splitting coherency for contributor organizations across foundations doesn’t seem like a good rationale either. Spin is devex and orchestration, and cncf already holds numerous projects to the goals and targeted towards wasm.

It’s not clear what benefit it has to users or community on the split but the risks to the same seem high unless spin is part of the same proposal to move to a foundation.

endocrimes commented 7 months ago

@devigned

I believe it is significant that the namesake project at the core of SpinKube, Spin, is itself not a CNCF project. My main concern regarding this is that if there was a change of licensing for Spin, then the SpinKube collection of projects could be at risk.

I understand that concern and it's one we're currently taking very seriously (sorry for the delay, I was on vacation post KubeCon and wanted to have some internal discussions before posting a response). We're just as committed to keeping Spin open as we have been from the beginning with the Open Source Promise. We're currently working on defining how best to ensure that Spin will remain open for everyone.

Mossaka commented 7 months ago

We're just as committed to keeping Spin open as we have been from the beginning with the Open Source Promise.

I praise the Fermyon team for pledging to keep Spin open sourced in a good faith commitment and the fact that the team has a track record of building open source tools and platforms. However, we must consider the possibility of an acquisition scenario that a new ownership may change the open-source license to the closed-source model. This scenario isn't unheard of. I am interested in hearing considerations on how to ensure that Spin will remain open in enforceable frameworks.

angellk commented 7 months ago

TAG App Delivery is also relevant to SpinKube - Our primary distribution mechanism for Wasm artifacts are OCI Artifacts, which ties into the work of wg-artifacts.

@endocrimes TAG App Delivery would love to understand more how SpinKube is using OCI Artifacts. Could you please present to and engage with the Artifacts WG?

CNCF slack: #wg-artifacts Meeting Info/Notes

devigned commented 7 months ago

@angellk I believe @jsturtevant and @thomastaylor312 will be leading a discussion about Wasm OCI Artifacts in the TAG-Runtime Wasm-WG in the upcoming meeting on Tuesday April 16th at 5:00 pm CET/ 8:00 am PST meeting notes / working group details.

For a preview, I believe this doc, WASI OCI Design, tells the gist of the story.

We pursued this in https://github.com/containerd/runwasi/pull/147 so that we could fulfill the promise of Wasm workloads being architecture and OS agnostic in Kubernetes. Previous to this work, one would need to build an OCI image for a given platform / arch (linux/arm, windows/amd64, ...) etc. To target multiple, you'd have to bake multi arch images. That is not Wasm. We believed that K8s Wasm workloads should be free to run on any node in your cluster with a single artifact.

IMHO, it's one of the coolest features of the containerd Wasm shims!

angellk commented 7 months ago

Thanks @devigned - please reach out to the TAG App Delivery Artifacts Working Group to make sure they are invited / involved then since they are specifically called out in the application.

endocrimes commented 7 months ago

For those that are interested outside of the context of this issue, we now have an upstream tracking issue in Spin for governance definition and are making good progress in figuring out the path forwards. Thanks for your patience 💖 : https://github.com/fermyon/spin/issues/2449

jsturtevant commented 7 months ago

Thanks @devigned - please reach out to the TAG App Delivery Artifacts Working Group to make sure they are invited / involved then since they are specifically called out in the application.

As a follow up, @devigned did an overview of the work at the last meeting on Tuesday April 12th https://docs.google.com/document/d/1E7iKPOuyA1jxPe8vDG8aPd8jtnCEbpDpCifXDvDCnA0/edit#heading=h.uylsvgzqikm

angellk commented 4 months ago

@rchincha @sabre1041 Does TAG App Delivery Artifacts WG have notes/recommendation from the meeting mentioned?

sabre1041 commented 4 months ago

@rchincha @sabre1041 Does TAG App Delivery Artifacts WG have notes/recommendation from the meeting mentioned?

Recording from the meeting which provided a brief overview and how OCI artifacts are being used

https://www.youtube.com/watch?v=NHW5UigLXso

dims commented 4 months ago

We're just as committed to keeping Spin open as we have been from the beginning with the Open Source Promise.

I praise the Fermyon team for pledging to keep Spin open sourced in a good faith commitment and the fact that the team has a track record of building open source tools and platforms. However, we must consider the possibility of an acquisition scenario that a new ownership may change the open-source license to the closed-source model. This scenario isn't unheard of. I am interested in hearing considerations on how to ensure that Spin will remain open in enforceable frameworks.

@endocrimes any response?

dims commented 4 months ago

@endocrimes poke!

kapilt commented 4 months ago

@endocrimes given zero movement on a foundation for spin core in the last quarter or two, the question must be asked, why the wrapper without the core in cncf? else splitting foundations is negative value for end users, and having a wrapper without a core in a foundation is the same. to be blunt, foundations aren't for laundering projects by startups, they are for fostering community around projects, imho.

endocrimes commented 4 months ago

@endocrimes poke!

We've actually been making some pretty meaningful progress - with Spin now having a defined governance model (https://github.com/fermyon/spin/pull/2593) and "spin" being a more well defined set of projects - as a very important step in the process of defining its future. We'll have more to share soon as that process settles in.

TheFoxAtWork commented 4 months ago

All - we appreciate community members support and exploration in surfacing items for technical consideration as part of sandbox applications, it allows the TOC to have a more comprehensive perspective on each projects' potential. However, in keeping with the Technical Leadership principles, we want to remind everyone to be welcoming and curious in their comments, provide feedback with courtesy, and foster respectful resolution. Embodying the technical leadership principles as you comment on these applications directly contributes to sustaining the open, community-oriented nature of CNCF.

We've initiated an update to the Sandbox Application README to capture this guidance for anyone commenting on applications (as well as providing more information to projects on what to expect).

raravena80 commented 3 months ago

Review from TAG-Runtime available here.

Note that there is a section of Technical Concerns and Moving Levels to Incubation Concerns

Thank you.

endocrimes commented 3 months ago

Spin has now also been submitted for separate consideration as a Sandbox project - It's taken a little while and a lot of discussions to find the right path for the health of the project, and a lot of work on cementing governance 😅.

The projects are being submitted separately as they have different design goals and aspirations, even though there is some overlap between the Shim projects in SpinKube and Spin itself. I strongly feel that is the best path for both of the projects futures over a subproject-like dynamic.

raravena80 commented 3 months ago

Hello, q. wouldn't it make sense to donate Spin first and then just do SpinKube as an add-on to Spin? Thanks.

TheFoxAtWork commented 3 months ago

Moving this project application to Waiting until Spin is up for review so they can be considered and reviewed at the same time.

rochaporto commented 3 months ago

Apologies for the delay, but for completeness here are the recommendations following the discussion.

@endocrimes the submission was reviewed by the CNCF TOC on August 13th. The following clarifications are requested:

The second item was answered by @endocrimes above already, and this is the sandbox application.

endocrimes commented 3 months ago

Would SpinKube consider supporting other WASM applications (WASMEdge, WASMCloud, ...), especially if Spin is not part of the application?

We’re open to supporting runtimes and application models outside of Spin.

We haven’t talked about this scenario in detail, but the design of the operator, runtime class manager, and the shim (the three major components of SpinKube) allow eventually supporting other execution environments.

More specifically, the operator has the concept of the “executor”, which can select the specific implementation of an executor which can control both the scheduling configuration for Kubernetes (https://github.com/spinkube/spin-operator/blob/main/api/v1alpha1/spinappexecutor_types.go), and also the actual execution of a Wasm component.

Similarly, the runtime class manager project already tracks a milestone for being able to configure all shims that KWasm can install (https://github.com/spinkube/runtime-class-manager/milestone/2)

In the near term, we are prioritizing being able to run more generic HTTP applications with SpinKube (this is also in line with the Spin roadmap, which aims to allow targeting runtimes other than the Spin runtime itself — https://github.com/fermyon/spin/blob/main/ROADMAP.md), and focusing on the operational experience for configuring and running them.

If others are interested in contributing and maintaining capabilities for SpinKube to support other Wasm applications, this is something the project can support, and we would definitely be open if it helps SpinKube users!