search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.16k stars 3.82k forks source link

changefeedccl: support for nats message queue #65845

Open amruss opened 3 years ago

amruss commented 3 years ago

NATS is a message queue/pubsub like system for publishing and subscribing to messages (https://nats.io/). NATS provides more fine-grained scalability and much simpler operational interface than Kafka at the expense of ordering and persistence.

NATS support as a CDC sink was requested by a user.

Epic CRDB-929

Jira issue: CRDB-7772

ajwerner commented 3 years ago

@stevendanna this is another reason that https://pkg.go.dev/gocloud.dev@v0.23.0/pubsub might have been handy for the gcloud support.

stevendanna commented 3 years ago

@ajwerner 👍 I believe we've resolved the main dependency issue that was making gocloud problematic, so if we pick that work back up I agree that using gocloud would be good.

danthegoodman1 commented 1 year ago

Would love to see this, esp with Jetstream adding persistence. Current workaround is to use the webhook destination and proxy to NATS

tamis-laan commented 3 months ago

Also very much interested in this!