search

konveyor / move2kube

Move2Kube is a command-line tool for automating creation of Infrastructure as code (IaC) artifacts. It has inbuilt support for creating IaC artifacts for replatforming to Kubernetes/Openshift.
https://move2kube.konveyor.io/
Apache License 2.0
383 stars 118 forks source link

feat: wasm transformers #1061

Open HarikrishnanBalagopal opened 1 year ago

HarikrishnanBalagopal commented 1 year ago

LFX link

https://mentorship.lfx.linuxfoundation.org/project/ec286a9e-e48d-4c83-a991-9c79a4ec213a

Move2Kube is a command-line tool for automating creation of Infrastructure as code (IaC) artifacts. It has inbuilt support for creating IaC artifacts for replatforming to Kubernetes/OpenShift. Move2Kube has a very plugin friendly architecture, users can write custom logic in the form of "Transformers" that Move2Kube can integrate seamlessly into its transformation pipeline. So far we have support for both Starlark and container image based transformers. We would like to support writing transformers as WASM modules that Move2Kube can run. WASM provides extensive sandboxing for security, it allows writing transformers in different language stacks like Rust, C/C++, etc. other than Golang, and WASM is just as lightweight and fast as Starlark.

Prerequisites

Please familiar yourself with the topics and try compiling simple Golang programs to WASI (preferably those that use functions imported from the host) with TinyGo so that you are familiar with the workflow and challenges involved.

Getting Started

Challenges

This is not an exhaustive list, just the ones we have found so far.

### Tasks
- [ ] Add support for custom transformers written in WASM
kanha-gupta commented 1 year ago

@HarikrishnanBalagopal any resources available to learn about wasm support ?

kmehant commented 1 year ago

@kanha-gupta You can start with WebAssembly Specifications, Hari will add more to this.

https://webassembly.org/specs/

HarikrishnanBalagopal commented 1 year ago

@HarikrishnanBalagopal any resources available to learn about wasm support ?

Have updated above. Can add more details if necessary.

mohamedawnallah commented 1 year ago

I'm really interested in this project. I wanted to provide you with an update on my progress with the Move2Kube WASM integration for custom transformers.

So far, I have successfully built the Move2Kube CLI and have been using it and (UI) to perform built-in and custom transformations as described in the Move2Kube documentation. Also, I've reported Inconsistent Planning Results Between Move2Kube UI and CLI bug in UI while following the Move2Kube tutorials. It has been an exciting experience so far, and I'm impressed with the plugin-friendly architecture that Move2Kube offers.

Currently, I am dedicating time to learn about TinyGo, WASM specifications, and how to compile Golang programs to the WebAssembly System Interface (WASI) API, with a particular focus on functions imported from the host.

I will let you know if any progress or questions about the project!

mohamedawnallah commented 1 year ago

Hi @HarikrishnanBalagopal @kmehant,

I wanted to share with you my recent experience with the Web Assembly using TinyGo. I took the time to understand the differences between TinyGo and Go Default Compiler and compiled a simple Golang program for WASI using TinyGo.

The program involved image processing for a checkerboard, and I imported functions from the host into JavaScript. Specifically, I configured the checkerboard size to be 1000 pixels x 1000 pixels and compared its performance with the vanilla JavaScript version. The results were impressive, as the computationally intensive task performed with Go and WebAssembly was nearly twice as fast as the JavaScript version. I have attached a demo for you to see where the website is refreshed every half second to illustrate the performance differences of checkboard generation.

https://github.com/konveyor/move2kube/assets/69568555/b616c965-e8aa-488f-b2db-a606627c5d80

Moreover, I noticed that the performance margin increased until it hit the threshold of the machine's memory, which highlights the efficiency of implementing such tasks in Go using WebAssembly.

After going through this experience, I have a solid understanding of why WebAssembly is an excellent option for handling computationally intensive tasks. It clearly demonstrates the advantages of combining TinyGo and WebAssembly, as they prove to be lightweight and remarkably fast together.

mohamedawnallah commented 1 year ago

@HarikrishnanBalagopal @kmehant Now I'd like to know where is the Move2Kube transformation engine source code for custom transformers located ?

DhruvSinghiitmandi commented 1 year ago

Hi @kmehant @HarikrishnanBalagopal I am really interested in this project and issue. How can i start learning and contributing to this issue? . So far I have built the move2kube CLI and tried the WASM and WASI TinyGO Tutorial ,currently I'm trying to figure out how Transformers and Custom Transformers function

HarikrishnanBalagopal commented 1 year ago

@mohamedawnallah @DhruvSinghiitmandi Thanks for your interest, please apply on the LFX portal https://mentorship.lfx.linuxfoundation.org/project/ec286a9e-e48d-4c83-a991-9c79a4ec213a You can use your findings to design a project proposal in the list of tasks in the LFX portal.

@HarikrishnanBalagopal @kmehant Now I'd like to know where is the Move2Kube transformation engine source code for custom transformers located ?

Have added the link to tutorial, code and documentation to the Getting started section in the first post.

Hi @kmehant @HarikrishnanBalagopal I am really interested in this project and issue. How can i start learning and contributing to this issue? . So far I have built the move2kube CLI and tried the WASM and WASI TinyGO Tutorial ,currently I'm trying to figure out how Transformers and Custom Transformers function

Have added some more links to the custom transformers docs and code in the Getting started section. If you have any specific questions regarding code/documentation please post it in this thread or ask in the #konveyor channel on the Kubernetes Slack workspace https://kubernetes.slack.com/archives/CR85S82A2

Srishti969 commented 1 year ago

Hi @kmehant @HarikrishnanBalagopal ! What will the selection process be for this project ? Will you be looking for the progress made on this specific project?

HarikrishnanBalagopal commented 1 year ago

Hi @kmehant @HarikrishnanBalagopal ! What will the selection process be for this project ? Will you be looking for the progress made on this specific project?

There is a task assigned to all candidates to design a technical proposal on how they will solve this issue. There may also be an interview with the shortlisted candidates.

Progress made on the issue may also be considered but not a requirement. We are only expecting candidates to have a good understanding of the issue, technologies involved, and the code base.

vishalrajofficial commented 1 year ago

I am Vishal Raj, a final year CSE student. The Konveyor: Move2Kube project for implementing WASM Transformers sounds like an interesting challenge. I'm excited to contribute to this CLI tool and support the running of WASM modules as custom transformers. My familiarity with Golang and knowledge of WASM/WASI will be valuable assets in tackling this project. I'm eager to explore this plugin-friendly architecture and leverage the extensive sandboxing and language versatility of WASM to enhance Move2Kube's capabilities. Let's work together to make Move2Kube even more powerful and efficient with custom WASM transformers!

DhruvSinghiitmandi commented 1 year ago

@kmehant @HarikrishnanBalagopal How is wasmer for loading and executing wasm modules(transformer) ?

SVKREP commented 11 months ago

Thanks for accepting me as a mentee in the LFX program. I am currently working on this.