Closed shravyak23 closed 4 years ago
Hello @shravyak23 ,
thanks for reaching us.
This is suspicious to me:
192.168.108.43
IP address is used for CQL login but 192.168.47.81
is shown in nodetool?
It seems to me that you either made a mistake while creating this issue or the IPs are misconfigured. If the former is the case, this seems like the IP address itself can not be contacted. If you were unable to log in and auth failed, the respective error message in CQL would look differently. This issue seems to be about not able to connect to that IP at all.
What Kubernetes deployment are you using? Do you use something which is crafted manually by you?
Hello @smiklosovic ,
Sorry my bad, there is no issue with kubernetes cluster.
[exxxxx@seskp1mgmt1 ~]$ kubectl exec -it cassandraserver-0 -n cass-sso-new bash
cassandra@cassandraserver-0:/$ nodetool status
Datacenter: dc1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns (effective) Host ID Rack
UN 192.168.108.26 230.35 KiB 256 100.0% 49a593cb-b8aa-4ff2-9ae0-38100f4f8111 rack1
cassandra@cassandraserver-0:/$ exit
exit
[exxxxxx@seskp1mgmt1 ~]$ kubectl exec -it cassandraserver-0 -n cass-sso bash
cassandra@cassandraserver-0:/$ nodetool status
Datacenter: dc1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID Rack
UN 192.168.47.81 239.54 KiB 256 ? f646c24d-926c-4402-aaac-9540ef47ec82 rack1
@shravyak23
{'192.168.108.43': error(111, "Tried connecting to [('192.168.108.43', 9042)].
I believe this has nothing to do with LDAP.
Do you have the access to the cassandra logs? Are there some errors?
@smiklosovic,
Thanks for your valuble inputs. I would re-verify by restarting the pod, try to do the ldap changes and will also check with my kubernetes team regarding ips as well.
Is it mandatory to use Network topology. Connected to CassandraCluster at cassandraserver-0:9042. [cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4] Use HELP for help. cassandra@cqlsh> desc system_auth
CREATE KEYSPACE system_auth WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'} AND durable_writes = true;
Error in log: Some of the log is archived which I am unable to fetch.
ERROR [main] 2020-07-28 05:18:17,833 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:18:47,205 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:18:58,458 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:19:39,027 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:20:20,030 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:21:00,429 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:21:41,020 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:22:09,416 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:22:22,010 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:23:02,319 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:23:42,883 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:23:59,024 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:24:23,429 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:25:04,126 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:25:44,765 CassandraDaemon.java:749 - Exception encountered during startup ERROR [main] 2020-07-28 05:26:13,637 CassandraDaemon.java:749 - Exception encountered during startup
@smiklosovic , I tried configuring in other cluster also to check if there is issue with the ips. there is no issue on kubernetes side but getting following error. I have placed ldap properties in $CASSANDRA_CONF since it is already set. I have also tried placing it in some other path and mentioned the path for it. In both the cases I am facing the same problem.
system.log ERROR [main] 2020-07-30 02:41:01,620 CassandraDaemon.java:749 - Exception encountered during startup org.apache.cassandra.exceptions.ConfigurationException: Unable to locate readable LDAP configuration file from system property cassandra.ldap.properties.file nor from $CASSANDRA_CONF/ldap.properties. at com.instaclustr.cassandra.ldap.configuration.LdapAuthenticatorConfiguration.parseProperties(LdapAuthenticatorConfiguration.java:111) ~[cassandra-ldap-3.11.4.jar:na] at com.instaclustr.cassandra.ldap.LDAPAuthenticator.validateConfiguration(LDAPAuthenticator.java:95) ~[cassandra-ldap-3.11.4.jar:na] at org.apache.cassandra.auth.AuthConfig.applyAuth(AuthConfig.java:108) ~[apache-cassandra-3.11.4.jar:3.11.4] at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:149) ~[apache-cassandra-3.11.4.jar:3.11.4] at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:132) ~[apache-cassandra-3.11.4.jar:3.11.4] at org.apache.cassandra.service.CassandraDaemon.applyConfig(CassandraDaemon.java:665) [apache-cassandra-3.11.4.jar:3.11.4] at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:609) [apache-cassandra-3.11.4.jar:3.11.4] at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:732) [apache-cassandra-3.11.4.jar:3.11.4]
: Unable to locate readable LDAP configuration file from system property cassandra.ldap.properties.file nor from $CASSANDRA_CONF/ldap.properties.
It means you have not set it properly, read the readme:
https://github.com/instaclustr/cassandra-ldap
https://github.com/instaclustr/cassandra-ldap#configuration
$ ./cassandra -f -Dcassandra.ldap.properties.file=/where/is/my/ldap.properties
You may check this logic:
It means that your $CASSANDRA_CONF is either not set / empty (it is null), or that file is there but it is not readable.
My bet is that $CASSANDRA_CONF is empty.
Please answer these questions before submitting your issue. Thanks!
What version of Cassandra are you using?
What version of Cassandra LDAP are you using?
What LDAP server you are using? Any specifics?
What did you do?
I deployed cassandra service with single pod over kubernetes cluster. I have downloaded cassandra-ldap jar file in the cassandra lib and ldap.properties in the cassandra conf . I have mentioned classpath for jar file in cassandra-env.sh and path to ldap properties in jvm.otions file. Made changes in cassandra.yaml file as given in Readme.md.
What did you expect to see?
What did you see instead?
cassandra@cassandraserver-0:/$ cqlsh --ssl -u admin -p admin@12345 Connection error: ('Unable to connect to any servers', {'192.168.108.43': error(111, "Tried connecting to [('192.168.108.43', 9042)]. Last error: Connection refused")})
If you are having connectivity related issues please share the following additional information
Describe your Cassandra cluster
please provide the following information
output of
nodetool status
before changes cassandra@cassandraserver-0:/$ nodetool status Datacenter: dc1Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns Host ID Rack UN 192.168.47.81 285.01 KiB 256 ? xxx-xxxxxx-xxxxxx-xxx rack1
Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless after changes done: cassandra@cassandraserver-0:/$ nodetool status /home/cassandra/cassandra-dump/bin/nodetool: 19: /home/cassandra/conf/cassandra-env.sh: $: not found Error: Could not find or load main class org.apache.cassandra.tools.NodeTool