Cannot achieve consistency level QUORUM

[aploetz]

Seeing this on some of our 3.11.2 nodes when trying to use the 3.11 plugin.

CassandraDaemon.java:708 - Exception encountered during startup
java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException:
Cannot achieve consistency level QUORUM

I'm guess it's trying to validate something with the default cassandra user? But on some nodes (4 of 6) this isn't a problem. Here's our system_auth definition:

CREATE KEYSPACE system_auth WITH replication =
  {'class': 'NetworkTopologyStrategy', 'dc1': '3', 'de1': '3'}

Any thoughts?

[smiklosovic]

Hi @aploetz

Thanks for reaching us.

I ll get back to you soon, I ll try to replicate your issue ASAP.

[smiklosovic]

Hi @aploetz

please try 3.11.4 again.

Be sure that you are using NTS as you do for system_auth and that nodes are boostrapped / joined properly. If you moved from SimpleStrategy to NTS by altering KS, be sure you ve done rebuild of that KS too.

The current 3.11.4 branch, I have tested this like:

1) I started 3 nodes in DC1 where system_auth had SimpleStrategy and RF1 2) I cqlsh-ed to node1 and altered KS to use NTS against dc1 with RF3 3) I did nodetool rebuild -ks system_auth 4) I started three other nodes in dc2, ootb, Cassandra sets up system_auth to have SimpleStrategy and rf 1 5) I cqlsh-ed to node1 and altered KS so I added dc2 with RF3 6) I did 3) 7) I was able to do cqlsh -u admin -p admin against whatever node

I am not sure how you are connecting that from your client, if via cqlsh or via driver or other mean. In that case try to "unset" your CL and use defaults first.

[smiklosovic]

@aploetz

I have released all branches to Maven Central for your convenience.

https://search.maven.org/search?q=a:cassandra-ldap

[aploetz]

FYI- We rebuilt our test cluster with steps 1-3, and saw the issue on one node. We then created a new node, and it joined ok; so I think this was just a bad node or two. Thanks for the help!

[aploetz]

Štefan,

You rock.

Thank you!

Aaron Ploetz

On Apr 2, 2019, at 10:53 PM, Štefan Miklošovič notifications@github.com wrote:

@aploetz

I have released all branches to Maven Central for your convenience.

https://search.maven.org/search?q=a:cassandra-ldap

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.