graphite-project / carbon

Carbon is one of the components of Graphite, and is responsible for receiving metrics over the network and writing them down to disk using a storage backend.
http://graphite.readthedocs.org/
Apache License 2.0
1.5k stars 490 forks source link

Pop RabbitMQ queue with carbon-cache #855

Closed kees-closed closed 5 years ago

kees-closed commented 5 years ago

Hi,

In carbon-cache there is the option to use AMQP, such as RabbitMQ. But the use isn't documented. I've setup the following on a RHEL7 system.

ENABLE_AMQP = True

AMQP_VERBOSE = True

AMQP_HOST = localhost
AMQP_PORT = 5672
AMQP_VHOST = /
AMQP_USER = graphite
AMQP_PASSWORD = pass
AMQP_EXCHANGE = graphite
AMQP_METRIC_NAME_IN_BODY = True

Below is the function I wrote to send data to the queue.

    def send_message(self, message):
        # Send the metrics, TLS not yet implemented due to configuration problems in RabbitMQ
        credentials = pika.PlainCredentials("graphite", "pass")
        parameters = pika.ConnectionParameters(self.server,
                                               5672,
                                               "/",
                                               credentials)#,
                                               #ssl=True,
                                               #ssl_options=dict(
                                               #                ssl_version=ssl.PROTOCOL_TLSv1_2,
                                               #                ca_certs="/usr/local/etc/pki/tls/certs/rabbitmq/ca_certificate.pem",
                                               #                certfile="/usr/local/etc/pki/tls/certs/rabbitmq/client_certificate.pem",
                                               #                keyfile="/usr/local/etc/pki/tls/certs/rabbitmq/client_key.pem",
                                               #                cert_reqs=ssl.CERT_REQUIRED))
        connection = pika.BlockingConnection(parameters)
        channel = connection.channel()
        channel.queue_declare(queue="graphite")
        channel.exchange_declare(exchange="graphite", exchange_type="topic")
        channel.basic_publish(exchange="graphite", routing_key="graphite", body=message)
        connection.close()

This does create the queue and the data is inserted, but carbon-cache doesn't extract the data. If the Graphite project can provide an example code and config with some documentation then that would be really helpful.

# rabbitmqctl list_queues
Listing queues
graphite        151
# rabbitmqctl list_exchanges
Listing exchanges
amq.rabbitmq.log        topic
amq.fanout      fanout
amq.direct      direct
amq.topic       topic
amq.headers     headers
graphite        topic
        direct
amq.rabbitmq.trace      topic
amq.match       headers
piotr1212 commented 5 years ago

I don't know anyone who is actually using carbon with AMQP. Is there nothing in your listener.log?

kees-closed commented 5 years ago

The problem was that the RabbitMQ user I used didn't have admin rights. Apparently it needed that in order to organize the queues.