datastax / metric-collector-for-apache-cassandra

Drop-in metrics collection and dashboards for Apache Cassandra
Apache License 2.0
111 stars 54 forks source link

"Nodes Status" panel shows single node cluster as down #101

Open jvf opened 11 months ago

jvf commented 11 months ago

The "Nodes Status" panel from the "Cassandra Overview" dashboard uses the following query to determine if a node is considered up:

max by (cluster, dc, rack, instance) (changes(mcac_thread_pools_completed_tasks{cluster=~"$cluster", dc=~"$dc", rack=~"$rack", instance=~"$node", pool_name="gossip_stage"}[2m:30s])) > bool 0

For single node clusters the GossipStage ThreadPool is always 0.

I know this is an edge case but having for example single-node test instances displayed as DOWN is annoying. Isn't there a better metric? nodetool status is showing single-node clusters as up, does someone know what metric they are using?

Nassz commented 10 months ago

Use up or another metric, as this Gossip Stage is unreliable even for bigger clusters.