search

apache / airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
https://airflow.apache.org/
Apache License 2.0
37.03k stars 14.28k forks source link

Nats Operator & Sensor #36446

Open khoramism opened 10 months ago

khoramism commented 10 months ago

Description

I think a lot of the teams already use nats as their message broker or streaming services and it would be a great Idea to have this as a part of the operators in airflow as well.

Use case/motivation

What I've been trying to achieve is the same process with the kafka, a Producer and a Consumer working together to implement or do something. For example in my case I tried to use airflow as an Alerting system with Kafka, I'd do some stuff using different operators and at the end of the day, I'd just push all the anomalies or errors to a topic in Kafka, then I'd use Kafka and telegram or slack operators to consume from the topic and send the messages needed to a specific channel or ...

And It would just be a great idea to have the same process and support out of the box for Nats as well as Kafka

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

boring-cyborg[bot] commented 10 months ago

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

Taragolis commented 10 months ago

Please note that all new Community supported providers should comes thought Accepting new community providers process.