Proposing Deno as an Alternative to Nodejs in Wing

skorfmann commented 1 year ago

Abstract

The objective of this issue is to contemplate the possibility of making Wing a truly autonomous programming language by replacing the prerequisite of having an active Nodejs setup with a custom Deno Runtime. I also foresee potential benefits of utilizing Deno as an inflight runtime.

Understanding Deno

Much like the Wing Compiler, Deno is fundamentally constructed in Rust and is dedicated to offering an enhanced user experience around TypeScript by significantly reducing the necessary boilerplate to run code. It also offers comprehensive security features. The critical aspect for us is its ability to serve as a base for custom runtimes. It's MIT licensed, which should mitigate licensing conflicts.

Deno's Architecture

Here's a brief summary of Deno's overall architecture. The captivating aspect is how the low-level functionality (system-level APIs), also known as OPS in Deno, are implemented. OPS are composed in Rust and can engage in bidirectional communication with V8 via a Rust bridge. The ability to create custom ops easily is a striking feature.

Deno's modular core allows the construction of custom runtimes. A well-known example is Deno Deploy, which powers cloud functions for entities like Slack, Supabase, and Netlify.

Two technical blogs discuss creating custom runtimes with Deno (blog 1 / 2).

A personal project of mine involved building a custom Cloud Runtime (which was my initial interaction with Rust). The project can be found here. It includes a CLI, a custom runtime package with some custom OPS, and a corresponding AWS Lambda runtime which is based on the AWS Lambda Rust Runtime.

NPM Support in Deno

Deno launched npm support in late 2022, followed by several improvements. There's also an npm registry client & resolver from Deno.

However, one downside in my project was the bundler, owing to the fact that the NPM package loader is a private package of the Deno CLI. The Deno team has indicated in their Discord that this is likely to improve (for instance, deno compile recently added support for npm).

It might be beneficial to engage with the Deno community to ascertain their plans around making this more accessible beyond the Deno CLI, and potentially contributing to this effort.

Having said that, I think it would still be possible to build this, even without further improvements in Deno itself.

Wing & Deno: Potential Synergy

Based on my limited understanding of the current Wing architecture, I see potential for both in-flight and pre-flight applications:

Preflight Runtime (e.g., synthesizing CDK Constructs)
Inflight Runtime (e.g., Custom AWS Lambda Runtime)
CLI

Preflight / Compiler

It's feasible to synthesize AWS CDK constructs with Deno without modifying the code, thanks to import maps. A simple live example is available.

A related discussion exists in the JSII repo.

I haven't tested this with CDK for Terraform, but I don't see any potential issues.

Incorporating an embedded Deno as part of Wing to synthesize constructs could enhance the current state by:

Eliminating the need for users to have a Nodejs installation, thereby strengthening the perception of Wing as a standalone platform.
Providing granular control over permissions for executed code, allowing for limited and controlled access to disk, environment, or network.
Supporting module imports via URL, which could be beneficial for applications like the Playground (unclear if / how this would work in a browser context)

CLI

The CLI could potentially be rewritten in Rust since all other parts should integrate smoothly. Alternatively, Deno compile could be used to encapsulate the existing CLI as a self-contained binary, akin to how the Deno Deploy CLI is constructed.

Inflight

The Deno core runtime could be embedded and extended via OPS as needed. OPS, as far as I understand, are not dynamically loadable and thus need to be present at compile time.

SDKs or Plugins could still be TypeScript modules/packages.

A key benefit of using Deno as a custom inflight runtime would be consistency across all potential targets. Additionally, this could lead to:

Less error-prone local development workflows, as the entire runtime would be owned by Wing.
Fine-grained control over which interfaces to expose to JS land.
The ability to provide custom, isolated, and secure extensions via OPS.

Concluding Thoughts

In my view, leveraging Deno instead of Nodejs could lead to Wing becoming a truly self-contained programming language. This change could simplify user requirements and provide additional security features. Further, by making use of Deno's ability to serve as a foundation for custom runtimes, we could potentially offer a more flexible and robust inflight runtime.

I would be more than happy about feedback and further discussions.

ShaiBer commented 1 year ago

Awesome proposal @skorfmann! I think it is definitely worth deeper investigation after the launch.

eladb commented 12 months ago

I love this direction. Definitely feels like there's huge synergy and will clean up our architecture quite beautifully.

@skorfmann one of the benefits of our current wasm-based approach is that we don't need any platform-specific binaries. IIUC if we use Deno and compile natively, we will need to ship a binary per platform, correct?

hasanaburayyan commented 12 months ago

It's feasible to synthesize AWS CDK constructs with Deno without modifying the code, thanks to import maps. A simple live example is available.

@skorfmann This seems super interesting. I have yet to dive into deno in any real depth, but totally out of curiosity taking a look at the deno.json what's the difference in these two aws-cdk-lib imports?

{
  "imports": {
    "aws-cdk-lib": "npm:aws-cdk-lib@2",
    "aws-cdk-lib/": "npm:/aws-cdk-lib@2/",
    "constructs": "npm:constructs@10"
  }
}

ShaiBer commented 12 months ago

This is the answer according to ChatGPT:

From the code snippet you provided, it appears you're defining import maps in the Deno configuration file (deno.json). Import maps are a standard feature in Deno that allow developers to specify custom resolutions for module imports. This is especially useful when you want to ensure consistency across all parts of your application and avoid potential versioning conflicts.

The entries for "aws-cdk-lib" and "aws-cdk-lib/" in your import map seem to be specifying different module resolutions:

"aws-cdk-lib": "npm:aws-cdk-lib@2"

This entry means that anywhere in your Deno code, if you use import { something } from 'aws-cdk-lib';, Deno will replace 'aws-cdk-lib' with the specified URL "npm:aws-cdk-lib@2" and import version 2 of the AWS CDK library from the npm package registry.
"aws-cdk-lib/": "npm:/aws-cdk-lib@2/"

This entry is a bit unusual and may be an error, as the correct syntax should be "aws-cdk-lib/": "npm:aws-cdk-lib@2/". This entry means that if you import a sub-module from the AWS CDK library in your code, like import { something } from 'aws-cdk-lib/aws-s3';, Deno will replace the base URL 'aws-cdk-lib/' with "npm:aws-cdk-lib@2/", and import the sub-module from version 2 of the AWS CDK library in the npm package registry.

Please note that at my last training cut-off in September 2021, Deno did not directly support the npm: scheme in import maps. It might be possible that Deno has added this feature in the meantime. If not, you would need to use a direct URL or a local file path.

It's also worth mentioning that the AWS CDK (Cloud Development Kit) is a software development framework for defining cloud infrastructure as code and provisioning it through AWS CloudFormation. The AWS CDK uses familiar programming languages, including TypeScript, JavaScript, Python, C#, and Java. The 'aws-cdk-lib' is a software package from AWS that contains all stable AWS Construct Library modules, including those for AWS CloudFormation, AWS IAM, and AWS Lambda.

skorfmann commented 12 months ago

"aws-cdk-lib": "npm:aws-cdk-lib@2" This entry means that anywhere in your Deno code, if you use import { something } from 'aws-cdk-lib';, Deno will replace 'aws-cdk-lib' with the specified URL "npm:aws-cdk-lib@2" and import version 2 of the AWS CDK library from the npm package registry.

👍

2. "aws-cdk-lib/": "npm:/aws-cdk-lib@2/" This entry is a bit unusual and may be an error, as the correct syntax should be "aws-cdk-lib/": "npm:aws-cdk-lib@2/". This entry means that if you import a sub-module from the AWS CDK library in your code, like import { something } from 'aws-cdk-lib/aws-s3';, Deno will replace the base URL 'aws-cdk-lib/' with "npm:aws-cdk-lib@2/", and import the sub-module from version 2 of the AWS CDK library in the npm package registry.

Don't remember exactly, but I think had issues when leaving out the first /.

skorfmann commented 12 months ago

@eladb

@skorfmann one of the benefits of our current wasm-based approach is that we don't need any platform-specific binaries. IIUC if we use Deno and compile natively, we will need to ship a binary per platform, correct?

Yes, since Deno is based on V8, bringing this to WASM is a challenge if possible at all. On the other hand, one could argue that the binary dependency of Wing is just outsourced to Nodejs at the moment. But in particular in the context of a browser not being able to use WASM in such a way could certainly be an issue.

github-actions[bot] commented 10 months ago

Hi,

This issue hasn't seen activity in 60 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days. Feel free to re-open this issue when there's an update or relevant information to be added. Thanks!

ekeren commented 10 months ago

Keep On Sat, Aug 5, 2023 at 09:04 github-actions[bot] @.***> wrote:

Hi,

This issue hasn't seen activity in 60 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days. Feel free to re-open this issue when there's an update or relevant information to be added. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/winglang/wing/issues/2741#issuecomment-1666405233, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANFVK7Z2ERX2PXI4Y5RSJDXTXPABANCNFSM6AAAAAAYWAFVRE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

guy-borderless commented 8 months ago

note this blog post by supabase, where they announce they plan to use the same characteristics of deno's architecture to build a runtime for api-gateway/reverse-proxy code. github issues for that are here.

skorfmann commented 8 months ago

note this blog post by supabase, where they announce they plan to use the same characteristics of deno's architecture to build a runtime for a api-gateway/reverse-proxy code. github issues for that are here.

That's super interesting, thanks for sharing!

Chriscbr commented 3 months ago

Another alternative: https://github.com/awslabs/llrt

eladb commented 3 months ago

MarkMcCulloh commented 3 months ago

I enjoy that wing's toolchain allows you to choose a runtime. I'm definitely not against having an opinioned default (embedded even) but it should still be easy to choose a different one. Deno is nice but their previously explicit goal of not being node-compatible could be a deal-breaker that I'd like to be able to opt out of.

Different runtimes for inflight are basically a requirement because I don't think it's reasonable to expect we'll always be able to guarantee that all targets can use the same runtime. I'd rather be able to choose my inflight runtime locally too to match that. e.g. if I'm deploying to cloudflare workers then I'd want to use workerd locally to do the simulation.

This is all separate from whether or not we choose to distribute wing as a native binary, we can (and eventually should) do that with or without an embedded runtime. That executable could even manage its own runtime installation for you, or just let you use your own global one. The same thing goes for package management btw.

On WASM: Only a subset of the toolchain really needs to work on WASM, and that's wingc. This allows us to do an embedded language server in the browser which I think is really valuable. So as long as we can still distribute that particular crate as a WASM binary then whatever we put on top can be natively compiled.

github-actions[bot] commented 2 weeks ago

Hi,

This issue hasn't seen activity in 90 days. Therefore, we are marking this issue as stale for now. It will be closed after 7 days. Feel free to re-open this issue when there's an update or relevant information to be added. Thanks!

winglang / wing