search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
96 stars 4.92k forks source link

RabbitMQ output #581

Closed Semyazz closed 6 years ago

Semyazz commented 8 years ago

Any plans to implement RabbitMQ (RMQ) output? I got isolated environments and I want to send everything through RMQ. I use cert-based auth and so on and it'd be awesome to utilize the same BUS here.

pikatoste commented 7 years ago

@warbaugh I'm also interested in the RMQ plugin.

Wernervdmerwe commented 7 years ago

+1

We use Graylog for a multitude of reasons. Graylog supports configuration management of both filebeat and NXLog, as well as supporting RabbitMQ as an intermediary. Graylog also supports pulling messages off a RabbitMQ queue, this is specifically useful for us having multiple servers in geographical different locations, both from not having multiple machines requiring access to the internet, as well as having to poke only one hole into a firewall and then the obvious reliability and security this provides.

relgames commented 7 years ago

hey @warbaugh, how is it going with RMQ plugin? Has it been released?

nasirus commented 7 years ago

+1

hbos commented 7 years ago

+1

ebuildy commented 7 years ago

Released on v6 => https://www.elastic.co/guide/en/beats/metricbeat/6.0/metricbeat-metricset-rabbitmq-node.html

Not for v5.

hbos commented 7 years ago

@ebuildy i think we want to use rabbitmq for transport, its not about the metrics.

sidleal commented 7 years ago

Hi all, I managed to send events to my RabbitMQ cluster using a custom output and MQTT plugin. More tests needed but it seems to work for me. And you need to clone and build your own version of the desired beat, so it's not so simple to use.

https://github.com/sidleal/mqttout

ebuildy commented 7 years ago

Doh @hbos :/ I -1 on me ^^

monicasarbu commented 6 years ago

Thank you, everyone, for the input. I am closing this issue because leaving it open was interpreted as a sign that we plan to implement a RabbitMQ output. Our position didn't change, but I'd like to clarify it.

Currently, we are not planning to add support for other outputs in Beats. We support Elasticsearch, Logstash, Redis and Kafka and we spend a considerable amount of time maintaining and supporting the outputs as we are trying to support the same features in all outputs, features like backpressure and guaranteed delivery. I think Logstash is doing an excellent job here supporting a lot of different inputs/outputs, and we would like not to double the effort to achieve the same thing.

I understand the concern in this thread that Beats require the installation of another component for sending to other outputs that we don't support, but we need to put that in balance with the scope of the project. We would like to concentrate our efforts in building features that make it easier to monitor your infrastructure, and to name a few: auto-discovery, monitoring your Docker containers, your Kubernetes pods, or just your RabbitMQ cluster.