Cloudflare Worker Compatibility/Integration

[nickchomey]

Proposed change

It currently isnt possible to use nats.js in Cloudflare workers due to some missing nodejs modules. Specifically fs and dns (at least from what I found - there could be more)

It appears that fs is only used if you specify TLS certs in the connection. But it doesn't appear to be possible to bypass dns.

In a conversation in Slack, a team member said that an option could be added to bypass dns resolution.

Additionally, he said that changes to/related to nuid.ts might be needed.

Use case

It would be great to be able to use NATS with Cloudflare Workers, so that connections could be made to NATS servers from the largest edge network.

One use case (among surely infinite) would be to use your NATS Jetstream KV instead of Cloudflare Workers KV, which has up to a 60 second propagation delay for KV writes while NATS KV is "immediately consistent".

I could also see use cases such as wanting to create a NATS connection from CF Workers, rather than directly from a browser client.

In fact, it would be SUPER cool if we could connect from, say, a browser to a CF Worker, which then opens and maintains a connection to a NATS server and subscribes to subjects/streams or watches KV changes. Then any changes could be returned to the browser via, say, an SSE keepalive connection. This would avoid the browser having to do anything with NATS, and could also make use of the CF worker to do any heavy processing prior and just returning, say, HTML to the browser.

It is perhaps possible that nats.ws could already be used with CF workers, but surely it makes most sense to use the native protocol.

More generally, Cloudflare's mission (I've seen it stated somewhere) seems to be to be a sort of operating system for the global internet. An integration with NATS, which makes it so simple to integrate different backend services, seems like a natural fit. Cloudflare could be the front-end/gateway to connect to a NATS-based backend infrastructure.

Contribution

I doubt I could provide much code, but would be happy to help test it if needed.

[nickchomey]

It might be relevant that CF Workers has a tcp sockets connect() api. They use it for creating database connectors and much more. Here's a relevant blog article as well https://blog.cloudflare.com/workers-tcp-socket-api-connect-databases/ which seems to address all of these issues (dns, tls etc..).

Perhaps that API could be used to make this a simpler process for you? It might not need any "development" beyond just providing a snippet (or small library, e.g. nats-cf.js) for connecting to NATS via CF Workers connect().

They also seem to be open to collaborating with seemingly competing products, such as Upstash, Turso and more, so would surely help you as well.

Finally, they're also proposing this api as a general standard https://github.com/wintercg/proposal-sockets-api

[nickchomey]

Their even newer Hyperdrive service seems like it might be great fit for all of this as well. It does two main things:

1. Maintains a set of regional database connection pools across Cloudflare’s network, so a Cloudflare Worker avoids making a fresh connection to a database on every request. Instead, the Worker can establish a connection to Hyperdrive (fast!), with Hyperdrive maintaining a pool of ready-to-go connections back to the database. This connection pooling will be forever-free.
2. Second, it understands the difference between read (non-mutating) and write (mutating) queries and transactions, and can automatically cache your most popular read queries: which represent over 80% of most queries made to databases in typical web applications. This will be charged when it is out of open beta.

I'm sure that this wouldn't be useful for every NATS implementation, but surely could be useful for things like KV operations?

Blog announcement: https://blog.cloudflare.com/hyperdrive-making-regional-databases-feel-distributed/ Docs: https://developers.cloudflare.com/hyperdrive/

[nickchomey]

I fiddled around for a while and finally got this to work after learning about telnet. Importing connect from nats throws all sorts of errors, but using connect from cloudflare:sockets seems to work just fine.

I ran a NATS docker server on a remote server, and connected to it in one local terminal with nats --server=remote.server.ip.address sub hello.nick

Then using cloudflare wrangler on my local machine I ran and triggered the worker via the browser. The worker connects to the remote nats server and sends the PUB message.

The subscriber on the local machine using nats-cli then receives the message successfully.

Obviously telnet is a disaster, so it would be great if cloudflare:sockets connect() could be integrated into nats.js, perhaps with some sort of option/flag to choose it vs the standard mechanisms. Surely not a big job!

Some docs that I found helpful: Setting up a new CF Worker Project and Wrangler Using CF Worker TCP Sockets

Also, once this has been integrated, it seems to me that adding a quick doc (and perhaps even blog post or video) on this would be useful as well! Likewise letting CF know about it so that they can it to their integration docs, which would bring more attention to NATS. I created a thread in their Discord server about this - hopefully someone will reply.

import { connect } from 'cloudflare:sockets';
//import { connect as natsconnect } from "nats";

export default {
    async fetch(request, env, ctx) {
        const nats = { hostname: "5.161.125.2", port: "4222" };
        let writer;

        try {
            const socket = connect(nats);
            writer = socket.writable.getWriter();

            const encoder = new TextEncoder();
            const message = "hello world";

            const encodedMessage = encoder.encode(message);
            const payload_size = encodedMessage.length;

            const encoded = encoder.encode(`PUB hello.nick ${payload_size}\r\n${message}\r\n`);
            await writer.write(encoded);

            return new Response(socket.readable, { headers: { "Content-Type": "text/json" } });
        } catch (error) {
            return new Response(error.message, { status: 500 });
        } finally {
            if (writer) {
                await writer.close();
            }
        }
    }
};

[aricart]

Just for capturing:

 wrangler dev
 ⛅️ wrangler 3.19.0
-------------------
✔ Would you like to help improve Wrangler by sending usage metrics to Cloudflare? … no
Your choice has been saved in the following file: ../../../../Users/aricart/Library/Preferences/.wrangler/metrics.json.

  You can override the user level setting for a project in `wrangler.toml`:

   - to disable sending metrics for a project: `send_metrics = false`
   - to enable sending metrics for a project: `send_metrics = true`
✘ [ERROR] Could not resolve "util"

    node_modules/nats/lib/src/mod.js:33:49:
      33 │     const { TextEncoder, TextDecoder } = require("util");
         ╵                                                  ~~~~~~

  The package "util" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "crypto"

    node_modules/nats/lib/src/mod.js:38:22:
      38 │     const c = require("crypto");
         ╵                       ~~~~~~~~

  The package "crypto" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "stream/web"

    node_modules/nats/lib/src/mod.js:49:32:
      49 │         const streams = require("stream/web");
         ╵                                 ~~~~~~~~~~~~

  The package "stream/web" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "net"

    node_modules/nats/lib/src/node_transport.js:40:22:
      40 │ const net_1 = require("net");
         ╵                       ~~~~~

  The package "net" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "tls"

    node_modules/nats/lib/src/node_transport.js:42:22:
      42 │ const tls_1 = require("tls");
         ╵                       ~~~~~

  The package "tls" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "path"

    node_modules/nats/lib/src/node_transport.js:43:28:
      43 │ const { resolve } = require("path");
         ╵                             ~~~~~~

  The package "path" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "fs"

    node_modules/nats/lib/src/node_transport.js:44:41:
      44 │ const { readFile, existsSync } = require("fs");
         ╵                                          ~~~~

  The package "fs" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "dns"

    node_modules/nats/lib/src/node_transport.js:45:20:
      45 │ const dns = require("dns");
         ╵                     ~~~~~

  The package "dns" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

✘ [ERROR] Could not resolve "util"

    node_modules/nkeys.js/lib/index.js:35:25:
      35 │     const util = require("util");
         │                          ~~~~~~
         ╵                          "./util"

  The package "util" wasn't found on the file system but is built into node.
  Add "node_compat = true" to your wrangler.toml file to enable Node.js compatibility.

╭──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [b] open a browser, [d] open Devtools, [l] turn off local mode, [c] clear console, [x] to exit                                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

✘ [ERROR] Failed to build

[aricart]

It's a severely limited node environment.

[nickchomey]

CF workers has compatibility for most of those modules, as described in this link. https://developers.cloudflare.com/workers/runtime-apis/nodejs/

If you add node_compat = true to your wrangler.toml file as they suggest, most of those errors go away. I'm away from my computer, but I believe it says the only one remaining is stream/web (which seems odd, given that they have support for the Streams API, as can be seen in that link as well as the successful usage of streams in the snippet I shared)

And, as described above, using the connect() function from cloudflare workers TCP socket mechanism rather than nats.js' connect() allowed me to successfully connect to NATS and send a telnet PUB message that was received by a subscriber via nats-cli.

So, I'm hopeful that you could add an option that allows us to bypass nats.js' connect() in favour of Cloudflare's

Though I guess we would still have issues deploying the script due to missing (perhaps unused) node dependencies in cloudflare - perhaps a lightweight nats-cf.js fork could work? Whatever the case, I don't think that much needs to be done to allow for cloudflare compatibility! I'd be happy to help test etc

[aricart]

That would require a whole new client transport - effectively requiring yet another javascript client.

With that said using the nats.ws - which is an ES compliant library, I was able to get it running with a single change in the library using denoflare. How to do it with wrangler is another exercise.

import { NatsConnection, connect } from '/Users/aricart/Dropbox/code/src/github.com/nats-io/nats.ws/src/mod.ts';
let nc : NatsConnection;

export default {
    async fetch(request: Request, env: any) {
        if(!nc || nc.isClosed()) {
            nc = await connect({ servers: ['wss://demo.nats.io:8443'] });
        }
        try {
            return new Response('connected to ' + nc.getServer());
        } catch (e) {
            return new Response(e.message)
        }
    },
}

[nickchomey]

Thank you!

I just confirmed that using compatibility_flags = [ "nodejs_compat" ] in wrangler.toml only throws an error for stream/web (which you can just comment-out since streams are already in globalThis.ReadableStream. Also, as it turns out, instead of using node_compat = true (which apparently is a legacy mechanism), you can set compatibility_flags = [ "nodejs_compat" ] in wrangler.toml Then if you add node: in front of all of the required modules (e.g. require("node:stream/web") you no longer get any of the errors listed above.

Either way, the following error is then shown

✘ [ERROR] service core:user:nats: Uncaught Error: Some functionality, such as asynchronous I/O, timeouts, and generating random values, can only be performed while handling a request.

    at index.js:154:27 in fillRandom
    at index.js:193:9 in setPre
    at index.js:173:14 in init
    at index.js:163:14 in Nuid
    at index.js:233:20 in node_modules/nats/lib/nats-base-client/nuid.js
    at index.js:18:50 in __require2
    at index.js:272:18 in node_modules/nats/lib/nats-base-client/core.js
    at index.js:18:50 in __require2
    at index.js:525:18 in node_modules/nats/lib/nats-base-client/util.js
    at index.js:18:50 in __require2

✘ [ERROR] MiniflareCoreError [ERR_RUNTIME_FAILURE]: The Workers runtime failed to start. There is likely additional logging output above.

If you modify fillRandom to the following, this error no longer happens - the issue being that crypto.getRandomValues() seems to not be available in node (I found many reports of this)

function fillRandom(a) {
    var _a;
    if ((_a = globalThis === null || globalThis === void 0 ? void 0 : globalThis.crypto) === null || _a === void 0 ? void 0 : _a.getRandomValues) {
        try {
            globalThis.crypto.getRandomValues(a);            
        } catch {
            _getRandomValues(a);    
        }
    }
    else {
        _getRandomValues(a);
    }
}

But then another error pops up. I assume that'll keep happening and have no idea how deep it'll go.

So, yeah, it seems like unless you folks really want to roll up your sleeves (or get collaboration from Cloudflare's team), this might not be worth your effort (unless you find it to be a strategically important priority!)

I will give the nats.ws library a try and report back - I'm averse to using websockets in the browser for various reasons, but I don't have any particular objection to using them in Cloudflare Workers.

Thanks again for your attention and help!

[nickchomey]

I get this error with the nats.ws code you provided.

✘ [ERROR] No matching export in "node_modules/nats.ws/esm/nats.js" for import "NatsConnection"

    src/index.js:18:9:
      18 │ import { NatsConnection, connect } from 'nats.ws';
         ╵          ~~~~~~~~~~~~~~

If I add export { NatsConnectionImpl as NatsConnection }; to the bottoms of nats.js, then that error goes away. But it then throws this error, presumably because the functions in that class aren't sufficiently exported?

[wrangler:err] TypeError: nc.isClosed is not a function
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/index.js:23:16)
    at __facade_modules_fetch__ (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:46:16)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at Object.next (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:50:11)
    at jsonError (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:30)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at __facade_invoke__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:63:9)
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:114:11)
[wrangler:inf] GET / 500 Internal Server Error (191ms)
[wrangler:err] TypeError: nc.isClosed is not a function
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/index.js:23:16)
    at __facade_modules_fetch__ (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:46:16)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at Object.next (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:50:11)
    at jsonError (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:30)
    at __facade_invokeChain__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:53:9)
    at __facade_invoke__ (/home/nick/apps/cloudflare/nats/node_modules/wrangler/templates/middleware/common.ts:63:9)
    at Object.fetch (/home/nick/apps/cloudflare/nats/src/.wrangler/tmp/bundle-OQTKzb/middleware-loader.entry.ts:114:11)

If I ignore NatsConnection altogether and use this code,

import { connect  } from 'nats.ws';

export default {
    async fetch(request, env) {

        nc = await connect({ servers: ['wss://demo.nats.io:8443'] });

        try {
            return new Response('connected to ' + nc.getServer());
        } catch (e) {
            return new Response(e.message)
        }
    },
} 

then i get this error sporadically:

✘ [ERROR] workerd/jsg/util.c++:281: error: e = kj/compat/http.c++:3300: failed: expected result == Z_OK || result == Z_BUF_ERROR || result == Z_STREAM_END; Decompression failed; result = -3;  with reason; ctx.msg = invalid stored block lengths

  stack:
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2875f0a
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2878dbd
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@287a8a4
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2896da0
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@2896fda
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@1f9b1f0
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@28b5a30
  /home/nick/apps/cloudflare/nats/node_modules/@cloudflare/workerd-linux-64/bin/workerd@21aad48;
  sentryErrorContext = jsgInternalError

✘ [ERROR] Uncaught (in promise) Error: internal error

✘ [ERROR] Uncaught (async) Error: internal error

other times I get this in the browser

If I use step debugging, the issue seems to come from these lines

const cp = this.transport.connect(srv, this.options);
await Promise.race([

Ultimately, I just don't think I can get this to work. I'm a php developer so this is all beyond my capabilities/knowledge. But if someone more competent is willing to roll up their sleeves (or guide me, but then they might as well just do it themselves...), then I really do think it should be possible to get it working for Cloudflare Workers!

[Hebilicious]

Thank you for your efforts @nickchomey , the JS world can be an endless rabbithole of stuff like that...

I believe this would be a really good thing to get working and the reason why it hasn't happened yet might be because the JavaScript community tends to prefer cloud services instead of self-hosting things, and NATS unfortunately doesn't have something like upstash redis (afaik) ; an HTTP client for Redis which makes it much easier to use Redis with non node.js environments. Unfortunately the HTTP client doesn't work with redis pub/sub, hence how I found-out about this thread.

I would love to have an official way to use NATS with cloudflare workers and will subscribe to this issue.

[nickchomey]

@Hebilicious the conversation continued somewhere in slack - I think the link is in this issue somewhere. We ultimately concluded that this would be very difficult to implement, and surely not any time soon.

For my own purposes, I will either use nats.ws straight from the browser or, more likely, add an SSE mechanism to this existing nats-caddy-bridge module for caddy.

https://github.com/sandstorm/caddy-nats-bridge/issues/3

This will allow various benefits

• browser will communicate over normal http, which can be intercepted by the browser service worker as well as cloudflare workers.
• Http has other benefits over websockets like simplicity and compression (though i think the nats protocol adds compression)
• minimize the actual client connections to nats to the 1 (or few) caddy instances, which will do NATS pubsub on the SSE clients' behalf
• I'll be able to more easily interact with other things that are going through the caddy server (im going to be using frankenphp for my php application)
• and other things that don't come to mind right now

Anyway, I hope this helps!

[gedw99]

great to see this. I too am interested in running workers on Cloudflare with a connection to NATS.

I concluded that CF workers is too restricted and so then went in the direction of exposing NATS over HTTP and SSE and HTTP3 Web Transports.

SO here's some links relating to all this that I hope is useful.

https://github.com/yomorun/yomo/tree/master/example/2-iopipe is a good example because NATS and Pipes relate to each other in terms of data flow patterns. Yomo uses http3 and thats why it's fast and has geo load balancing. Can we run Yomo as a CF Worker was then where I wanted to head.

https://github.com/syumai/workers allows running golang works on CF workers.

https://github.com/tractordev/wanix allows running your worker in a browser also over http3 etc.

https://github.com/Gianfranco753/caddy-nats-bridge allows running your worker on a sever

SO you can see where I am going with this... NATS over HTTP3, so that we can run workers on CF, Browsers, servers.

The synergy is rather obvious in the feature section: https://github.com/syumai/workers?tab=readme-ov-file#features. OMG that looks a lot like NATS endpoints :)

[nickchomey]

@gedw99 im having trouble making sense of all of this. It seems like it all spans numerous issues - running nats client in cloudflare worker (this issue), caddy nats bridge module (the issue you opened in another repo that I linked to), another fork of the caddy nats bridge, wasm, and more.

I'm sure there's a lot of great ideas here, so it would be helpful if you could both organize them a bit more, suggest what goes where, and, most of all, let us know what you've successfully implemented.

As I mentioned in my previous comment, I'm now most interested in running a Caddy nats sse server/bridge, which (I think) would obviate the need for any of this fancy stuff with CF workers - just create an sse connection between the browser and caddy, which could be proxied through CF workers' native http support.

If that's something that you've already had some success with, or would like to help me build, perhaps we can move the conversation over to the caddy nats bridge issue you created and I linked to?

sandstorm/caddy-nats-bridge#3

[gedw99]

Hey @nickchomey

Yeah I can see how it might be confusing and yes I am happy to discuss over at https://github.com/sandstorm/caddy-nats-bridge/issues/3

Hopefully this helps explains the gist of why I had those 3 links. Forgive me for presuming and of course you and I probably have slightly differing agendas / intentions.

It shows how to run a NATS bridge over HTTP 1, 2, 3 / SSE and / or HTTP 3 Web Transport. HTTP 3 Web Transport is preferred for UDP perf reasons, but we need SSE too because of Apple keeping HTTP3 Web Transport behind feature flags ( for now ). If you study Yomo you can see that they also have fallbacks and have presence and a few other important things.

Then once you have that you have a NATS bridge like above you then have 3 places to run your workers via the code I linked to:

1. On your own server. Probably behind caddy NATS bridge, and possibly as a WASM worker.
2. On CF as a worker as a wasm worker.
3. In a Browser as a WASM worker.

Those 3 places are really 3 layers of caching. We know that NATS and CQRS are synergistic patterns. So events flow from the centre ( your server ) producing material data views using WASM workers. On top of that though, you can decide where you want those wasm workers to live and so hence where you want the Materialised data views to live.

So really all I am suggesting is that Caddy NATS bridge needs to incorporate HTTP 1, 2, 3, and http3 web transport. HTTP / SSE is definitely also needed, and then later HTTP3 Web Transport makes sense too.

[nickchomey]

Most of all, I'm just not sure how any of this relates to running a NATS client in a Cloudflare Worker, which is what this issue is about. Are you perhaps mistaken that this is the nats.js repo rather than the caddy nats bridge repo?

The only part that seems relevant is the prospect of running nats in a CF worker via wasm. Though I wonder if that's at all possible given that the CF worker runtime doesn't have the requisite node APIs. Have you tried this at all?

Anyway, I'm quite keen to work on caddy nats bridge. I should be able to start this week and am eager to learn more about all of this stuff and perhaps implement it there. Why don't we carry on over there?

[gedw99]

@nickchomey sure we can touch base over there.
You can contact me via the link on my GitHub profile if you want too.

[gedw99]

I use nats.go here for CF

It's a severely limited node environment.

yes

https://github.com/syumai/workers with nats.go works as wasm. tinygo is needed to keep the size tiny to stay in there CF site limits

its def a nice way to write workers.

[nickchomey]

@gedw99 please share some details on how to implement this!

[gedw99]

@gedw99 please share some details on how to implement this!

I don’t have it working but the skeleton code is there to be able to adapt nats.go into it because nats.go also compiles to WASM using tinygo. I know this because I have used nats.go that way.

https://github.com/syumai/workers/tree/main/_examples/sockets Looks like a decent path finder to work with nats.