Does C* exporter need to be run on every machine where C* is running ?

[shamimgeek]

Does Cassandra exporter need to be run on every machine where cassandra is running ? or it can be run on one machine and can connect to all nodes ?

How does it work. Please guide me.

[erebe]

Hello,

You need one exporter per cassandra node and after that you need to configure your prometheus to scrap all exporters.

Behind the hood, the exporter use Java JMX in order to retrieves metrics from the nodes. JMX is more than just for exposing metrics, that's a full featured RPC mechanism. Trying to attach one exporter to all instances at the same time does not scale well.

[deimosfr]

We will add this information in the README to get it clear

[shamimgeek]

@erebe and @deimosfr : Thanks for your reply

[gufranmmu]

If we enable remote JMX, the exporter doesn't necessarily need to be run in the same node, is that correct? :)

[deimosfr]

No, you have to. Because some metrics are node specific and you need to sum them to get a correct cluster overview. That's what we're providing in the grafana dashboard #13 (in progress)

[gufranmmu]

Thank you deimosfr! You guys are very active :) We ditch JMX exporter in favor of Cassanra_exporter :) In a mesos or kubernetes cluster, how do you know which agents are running Cassandra nodes and then launch the cassandra_exporter containers only in those agents?

[deimosfr]

I'm using it with Kubernetes this way https://github.com/MySocialApp/kubernetes-helm-chart-cassandra/tree/master/kubernetes/templates. If you're looking at the statefulset, you'll see 2 containers in a pod. 1 is cassandra and the other is the exporter targeting the cassandra instance in the pod.

As one agent is exporting a stateful instance name, in grafana, I'm displaying this name. If you look at prometheus, you'll get the ip instance name as well. If you think it's too complicated, I think I can add something in the grafana dashboard to display instance ip or cassandra statefulset name.

Does it answer your question?

[ambrons]

That's a slick solution @deimosfr, unfortunately that can not be used with docker swarm. I was pondering possible solutions, but I'd end up needing to add something to use tasks.serviceName to get a list of IPs and the use something like zookeeper or the like to store which cassandra_exporter is already monitoring which cassandra replica. I think in my case we're likely going to run it in the cassandra docker for simplicity.

[deimosfr]

I really don't know docker swarm and looks like too specific for me. But I was wondering if what you see in nodetool status command is the same that what you see with the ip shown in docker swarm? If yes I can try something

[deimosfr]

Any news?

[erebe]

feel free to re-open if needed