cloudflare / workerd

The JavaScript / Wasm runtime that powers Cloudflare Workers
https://blog.cloudflare.com/workerd-open-source-workers-runtime/
Apache License 2.0
5.87k stars 257 forks source link

investigate: can we polyfill node:net#Socket with cloudflare:socket? #2129

Open IgorMinar opened 1 month ago

IgorMinar commented 1 month ago

We need to investigate how hard is it to build a realistic node:net#Socket polyfill using what we currently have in workerd's cloudflare:socket and nodejs_compat layer.

This came out of a discusstion about https://github.com/brianc/node-postgres/pull/3170 with @petebacondarwin this morning where we are trying to improve our pg specific node:net#Socket polyfill. This made me wonder if a better path wouldn't be to beef up our node:net#Socket polyfill so that things just work.

Looking at https://github.com/brianc/node-postgres/blob/master/packages/pg-cloudflare/src/index.ts it seems that we are essentially creating a bespoke polyfill for node Socket there. So why not upstream this to unenv (or eventually workerd)?

The unenv polyfill doesn't seem terribly complex: https://github.com/unjs/unenv/blob/main/src/runtime/node/net/socket.ts because it simply extends from node:stream#Duplex.

Note: workerd has native implementation of node:stream#Duplex but since this unenv polyfill exports it via a relative import, the unenv polyfill will end up ignoring our implementation. This is something that should be fixed in unenv as part of the hybrid polyfill effort: https://github.com/unjs/unenv/issues/181

jasnell commented 1 month ago

We ought to be able to get very close. There will likely be some edge cases or options that we're not able to support, but for the most part it should be doable

IgorMinar commented 1 month ago

notes from a meeting:

Main concerns:

Estimate: 1 week of work

hoodmane commented 1 month ago

This would be great from the point of view of our idea of shimming Python socket with our socket. If we make the Python socket library work on top of node sockets then we can upstream it, potentially all the way into Emscripten.