The problem

NATS Streaming is a delight to use in development, but less so to deploy and operate (especially when using Kubernetes + GitOps):

It requires two backing services: NATS server and a fault-tolerant, persistence backend. That means: Increased costs (labour + hosting), and more components that could fail.
I'm having to deploy a highly-available PostreSQL instance just for NATS Streaming, which is insane. We already use MongoDB, but it isn't supported. (MySQL and a distributed file storage are also supported, but neither offers any advantage over PostgreSQL)
I'm uncomfortable using the official Helm charts in production as I don't think they're production-ready, based on the issued I've found (and submitted PRs for), the workarounds I've had to implement, the overall brittleness of the resources in the chart (e.g., provisioning DB tables via the initdb script) and the overall insecurity of the chart (e.g., storing sensitive values in the clear instead of using secrets).

This is what the systems architecture looks like today:

original

The solution

[x] https://github.com/relaycorp/relaynet-internet-gateway/issues/929
[x] https://github.com/relaycorp/relaynet-internet-gateway/issues/930
[x] Provision and integrate GCP resources (e.g., PubSub, Redis).
[x] Deprovision GCP and Kubernetes resources used by NAT Streaming (e.g., PostgreSQL).

This is what it'll look like afterwards:

after

Kafka is too expensive at +$1.2k/month for the simplest possible HA cluster. Kafka also feels like a massive overkill for what it'd be used for.