Open voriol opened 2 years ago
javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
The logs indicate a plugin configured as so (newlines added for readability):
<LogStash::Inputs::Kafka
topics=>["topic1", "topic2", "topic3"],
ssl_truststore_location=>"/opt/logstash/kafka/truststore.jks",
ssl_truststore_password=><password>,
kerberos_config=>"/opt/logstash/kafka/krb5.conf",
jaas_path=>"/opt/logstash/kafka/KafkaClient_jaas.conf",
bootstrap_servers=>"server1:9093,server2:9093,server3:9093",
client_id=>"logstash-i1",
decorate_events=>"extended",
codec=> <LogStash::Codecs::Plain id=>"plain_101e4499-6300-46ff-8740-84f845a66fc4", enable_metric=>true, charset=>"UTF-8">,
group_id=>"consumer1",
security_protocol=>"SASL_SSL",
id=>"c474ee0a252c935096ef4baa88631715b80e466ea26a4b02e464cefb9627dfdc",
sasl_kerberos_service_name=>"kafka",
enable_metric=>true,
connections_max_idle_ms=>540000,
metadata_max_age_ms=>300000,
request_timeout_ms=>40000,
schema_registry_validation=>"auto",
auto_commit_interval_ms=>5000,
check_crcs=>true,
client_dns_lookup=>"default",
consumer_threads=>1,
enable_auto_commit=>true,
fetch_max_bytes=>52428800,
fetch_max_wait_ms=>500,
heartbeat_interval_ms=>3000,
isolation_level=>"read_uncommitted",
key_deserializer_class=>"org.apache.kafka.common.serialization.StringDeserializer",
max_poll_interval_ms=>300000,
max_partition_fetch_bytes=>1048576,
max_poll_records=>500,
receive_buffer_bytes=>32768,
reconnect_backoff_ms=>50,
retry_backoff_ms=>100,
send_buffer_bytes=>131072,
session_timeout_ms=>10000,
value_deserializer_class=>"org.apache.kafka.common.serialization.StringDeserializer",
poll_timeout_ms=>100,
ssl_endpoint_identification_algorithm=>"https",
sasl_mechanism=>"GSSAPI"
>
As the logs state, upon establishing a connection, the kafka client was asked for a password but was unable to get one from you. It is possible that the jaas file you have provided does not contain the credentials, or is not discoverable by the user running the logstash process?
Hi @yaauie, thanks for your response.
The thing is: same config files, different logstash version, different behavior.
I expect: same config files, different logstash version, same behavior.
Thanks.
Ie, this works:
docker run --rm -it \
-v /tmp/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
-v /tmp/kafka:/usr/share/logstash/kafka/ \
docker.elastic.co/logstash/logstash:8.3.3
This not:
docker run --rm -it \
-v /tmp/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
-v /tmp/kafka:/usr/share/logstash/kafka/ \
docker.elastic.co/logstash/logstash:8.5.2
The content of /tmp/logstash.conf:
input {
kafka {
codec => json
topics => ["topic_name"]
group_id => "consumer_name"
bootstrap_servers => "server1:9093,server2:9093,server3:9093"
security_protocol => "SASL_SSL"
sasl_kerberos_service_name => "kafka"
jaas_path => "kafka/KafkaClient.conf"
ssl_truststore_location => "kafka/truststore.jks"
ssl_truststore_password => "xxxxxxxx"
}
}
output {
stdout { codec => rubydebug { metadata => true } }
}
The content of "/tmp/kafka":
/tmp/kafka/truststore.jks
/tmp/kafka/user.keytab
/tmp/kafka/KafkaClient.conf
The content of /tmp/kafka/KafkaClient.conf:
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="kafka/user.keytab"
principal="user@domain";
};
Logstash information:
Please include the following information:
Plugins installed: (
bin/logstash-plugin list --verbose
)JVM (e.g.
java -version
):If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:
java -version
)LS_JAVA_HOME
environment variable if set.OS version (
uname -a
if on a Unix-like system):Description of the problem including expected versus actual behavior:
Steps to reproduce:
Please include a minimal but complete recreation of the problem, including (e.g.) pipeline definition(s), settings, locale, etc. The easier you make for us to reproduce it, the more likely that somebody will take the time to look at it.
Provide logs (if relevant):