search

robinhood / faust

Python Stream Processing
Other
6.72k stars 535 forks source link

Implement Rabbit/AMQP Channel #373

Closed sivy closed 4 years ago

sivy commented 5 years ago

Some of us have existing RabbitMQ/AMQP implementations that would benefit from being migrated/extended into a Kafka environment; it would be helpful is there was a sandard (if limited) Rabbit/AMQP channel that we could use to consume message from Rabbit and publish them into Kafka (and vice versa).

joekohlsdorf commented 4 years ago

Could you explain why existing RabbitMQ to Kafka bridges don't work for you? We should really try our hardest to avoid making a mess comparable to Celery.

gsemet commented 4 years ago

I would vote for rabbitmq input / output as well, setting up a kafka cluster is pretty heavy and since we already have a reliable AMQP cluster, we would want to connect it to faust to do data streaming