search

pingcap / tiflow

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
Apache License 2.0
426 stars 283 forks source link

Support NATS / LIftBridge #201

Open winwisely99 opened 4 years ago

winwisely99 commented 4 years ago

Description

We cant run kafka as we just dont like the legacy of java.

It would be awesome if you also support LIftBridge. It has a GRPC interface as well as a NATS interface. Its basically like Kafka but written in Golang

https://github.com/liftbridge-io/liftbridge

Value Score

Optional value: 1~5

Workload Estimation

1 Point for 1 Person/Work Day

overvenus commented 4 years ago

This is a good feature request! We will consider it in a future release.

joe-getcouragenow commented 4 years ago

Aany news on this ?

Liftbridge hit v1.0 and we run it on about 20 servers and very happy with it.

How can this be moved forward ?

The golang client to send a message to liftbridge is in GitHub. Here is an example: https://github.com/liftbridge-io/go-liftbridge/blob/master/example/lift-pub/publish.go

I expect it will be pretty easy to just copy the kafka-consumer.go and make a liftbridge-consuner.go

Liftbridge also has an async publish command now too, so it eases the load on ticdc

Anyways please consider this

overvenus commented 4 years ago

We'd like to connect to Liftbridge! It's sound like a good first issue for contributors! I have labeled this issue help-wanted, any contribution is appreciated!

joe-getcouragenow commented 4 years ago

@overvenus thanks for this. If i find time i will pick it up here.

gerardwebb commented 3 years ago

@overvenus actually "nats jetstream" does most of what liftbridge does now. Both use Raft for replication of the queues for want of a better word.

https://github.com/nats-io/jetstream

cawfeecoder commented 2 years ago

What's the status of this feature @overvenus