search

fluent / fluent-plugin-kafka

Kafka input and output plugin for Fluentd
Other
303 stars 176 forks source link

fluentd-plugin-kafka supoort kerberos? #220

Open openchung opened 5 years ago

openchung commented 5 years ago

I use fluentd to send json log to sasl_ssl cloudera kafka , but I meet the following warn. So it cause send failed. I have verified my keytab and principal using kinit to verify .

2018-10-17 07:45:10 +0800 [warn]: temporarily failed to flush the buffer. next_retry=2018-10-17 07:47:30 +0800 error_class="GSSAPI::GssApiError" error="gss_init_sec_context did not return GSS_S_COMPLETE" plugin_id="object:3f84a1db517c" 2018-10-17 07:45:10 +0800 [warn]: suppressed same stacktrace 2018-10-17 07:45:10 +0800 fluent.warn: {"message":"Send exception occurred: gss_init_sec_context did not return GSS_S_COMPLETE"} 2018-10-17 07:45:10 +0800 fluent.warn: {"message":"Exception Backtrace : /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/gssapi-1.2.0/lib/gssapi/simple.rb:95:in init_context'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/sasl/gssapi.rb:72:ininitialize_gssapi_context'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/sasl/gssapi.rb:25:in authenticate!'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/sasl_authenticator.rb:51:inauthenticate!'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/connection_builder.rb:27:in build_connection'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/broker.rb:184:inconnection'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/broker.rb:170:in send_request'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/broker.rb:44:infetch_metadata'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:386:in block in fetch_cluster_info'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:381:ineach'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:381:in fetch_cluster_info'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:367:incluster_info'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:95:in refresh_metadata!'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/cluster.rb:50:inadd_target_topics'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/producer.rb:276:in deliver_messages_with_retries'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/producer.rb:238:inblock in deliver_messages'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/instrumenter.rb:23:in call'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/instrumenter.rb:23:ininstrument'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/ruby-kafka-0.6.8/lib/kafka/producer.rb:231:in deliver_messages'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-kafka-0.7.9/lib/fluent/plugin/out_kafka_buffered.rb:281:indeliver_messages'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-kafka-0.7.9/lib/fluent/plugin/out_kafka_buffered.rb:344:in write'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.40/lib/fluent/buffer.rb:354:inwrite_chunk'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.40/lib/fluent/buffer.rb:333:in pop'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.40/lib/fluent/output.rb:342:intry_flush'\n/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-0.12.40/lib/fluent/output.rb:149:in `run'"} 2018-10-17 07:45:10 +0800 fluent.info: {"message":"initialized kafka producer: kafka"} 2018-10-17 07:45:10 +0800 fluent.warn: {"next_retry":"2018-10-17 07:47:30 +0800","error_class":"GSSAPI::GssApiError","error":"gss_init_sec_context did not return GSS_S_COMPLETE","plugin_id":"object:3f84a1db517c","message":"temporarily failed to flush the buffer. next_retry=2018-10-17 07:47:30 +0800 error_class=\"GSSAPI::GssApiError\" error=\"gss_init_sec_context did not return GSS_S_COMPLETE\" plugin_id=\"object:3f84a1db517c\""}

Following is my td-agent.conf 

<system>
  log_level debug
</system>
<source>
  @type tail
  format json
  read_lines_limit 200
  path /mnt/old/tx-failover.log*
  pos_file /opt/kafka_failover.log.pos
  tag audit.trail
</source>
<match audit.*>
  @type kafka_buffered
  brokers 192.168.5.129:9093
  client_id kafka
  buffer_type memory
  default_topic log4j
  output_data_type json
  ssl_ca_cert /opt/server.cer.pem
  #sasl__mechanism gssapi
  principal kafka@KAFKA.COM
  keytab /opt/kafka.keytab
</match>
<match **>
  @type stdout
  output_type json
</match>

Following is my dependency actionmailer (4.2.8) actionpack (4.2.8) actionview (4.2.8) activejob (4.2.8) activemodel (4.2.8) activerecord (4.2.8) activesupport (4.2.8) addressable (2.5.2, 2.5.1) arel (6.0.4) aws-sdk (2.10.45) aws-sdk-core (2.10.45) aws-sdk-resources (2.10.45) aws-sigv4 (1.0.2) base91 (0.0.1) bigdecimal (default: 1.2.4) bson (4.1.1) builder (3.2.3) bundler (1.14.5) celluloid (0.15.2) cool.io (1.5.1) diff-lcs (1.3) draper (1.4.0) elasticsearch (5.0.5) elasticsearch-api (5.0.5) elasticsearch-transport (5.0.5) erubis (2.7.0) excon (0.62.0) faraday (0.13.1) ffi (1.9.25) fluent-logger (0.7.1) fluent-mixin-plaintextformatter (0.2.6) fluent-plugin-elasticsearch (1.17.2) fluent-plugin-genhashvalue (0.04) fluent-plugin-kafka (0.7.9, 0.6.1) fluent-plugin-mongo (0.8.1) fluent-plugin-rewrite-tag-filter (1.5.6) fluent-plugin-s3 (0.8.5) fluent-plugin-scribe (0.10.14) fluent-plugin-td (0.10.29) fluent-plugin-td-monitoring (0.2.3) fluent-plugin-webhdfs (0.7.1) fluentd (0.12.40) fluentd-ui (0.4.4) font-awesome-rails (4.7.0.1) globalid (0.4.0) gssapi (1.2.0) haml (4.0.7) haml-rails (0.5.3) hike (1.2.3) hirb (0.7.3) http_parser.rb (0.6.0) httpclient (2.8.2.4) i18n (0.8.1) io-console (default: 0.4.3) ipaddress (0.8.3) jbuilder (2.6.3) jmespath (1.3.1) jquery-rails (3.1.4) json (default: 1.8.1) kramdown (1.13.2) kramdown-haml (0.0.3) loofah (2.0.3) ltsv (0.1.0) mail (2.6.4) mime-types (3.1) mime-types-data (3.2016.0521) mini_portile2 (2.3.0, 2.1.0) minitest (5.10.1, default: 4.7.5) mixlib-cli (1.7.0) mixlib-config (2.2.4) mixlib-log (1.7.1) mixlib-shellout (2.2.7) mongo (2.2.7) msgpack (1.1.0) multi_json (1.12.1) multipart-post (2.0.0) nokogiri (1.8.1) ohai (6.20.0) oj (2.18.1) parallel (1.8.0) psych (default: 2.0.5) public_suffix (3.0.0, 2.0.5) puma (3.8.2) rack (1.6.5) rack-test (0.6.3) rails (4.2.8) rails-deprecated_sanitizer (1.0.3) rails-dom-testing (1.0.8) rails-html-sanitizer (1.0.3) railties (4.2.8) rake (default: 10.1.0) rdoc (default: 4.1.0) request_store (1.3.2) ruby-kafka (0.6.8) ruby-progressbar (1.8.3) rubyzip (1.2.1, 1.1.7) sass (3.2.19) sass-rails (4.0.5) settingslogic (2.0.9) sigdump (0.2.4) sprockets (2.12.4) sprockets-rails (2.3.3) string-scrub (0.0.5) sucker_punch (1.0.5) systemu (2.5.2) td (0.15.2) td-client (0.8.85) td-logger (0.3.27) test-unit (default: 2.1.10.0) thor (0.19.4) thread_safe (0.3.6) thrift (0.8.0) tilt (1.4.1) timers (1.1.0) tzinfo (1.2.3) tzinfo-data (1.2017.2) uuidtools (2.1.5) webhdfs (0.8.0) yajl-ruby (1.3.0) zip-zip (0.3)

Please help me.

openchung commented 5 years ago

update about my issues. I change to version 0.8.0 gem still have error. Following is the stacks :

`2018-10-21 12:32:38 +0800 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-elasticsearch' version '1.17.2' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-genhashvalue' version '0.04' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-kafka' version '0.8.0' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-mongo' version '0.8.1' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.6' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-s3' version '0.8.5' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-scribe' version '0.10.14' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-td' version '0.10.29' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.3' 2018-10-21 12:32:38 +0800 [info]: gem 'fluent-plugin-webhdfs' version '0.7.1' 2018-10-21 12:32:38 +0800 [info]: gem 'fluentd' version '0.12.40' 2018-10-21 12:32:38 +0800 [info]: adding match pattern="audit.*" type="kafka_buffered" 2018-10-21 12:32:38 +0800 [trace]: registered output plugin 'kafka_buffered' 2018-10-21 12:32:38 +0800 [info]: brokers has been set directly: ["192.168.5.129"] 2018-10-21 12:32:38 +0800 [info]: adding match pattern="**" type="stdout" 2018-10-21 12:32:38 +0800 [info]: adding source type="tail" 2018-10-21 12:32:38 +0800 [info]: using configuration file:

log_level trace 
@type tail
format json
read_lines_limit 200
path /mnt/old/tx-failover.log
pos_file /opt/kafka_failover.log.pos
tag audit.trail

<match audit.*> @type kafka_buffered brokers 192.168.5.129 client_id kafka buffer_type memory default_topic log4j output_data_type json ssl_ca_cert /opt/ca_cert.pem get_kafka_client_log true principal kafka/cipkafka1t.testesunbank.com.tw@KAFKA.COM keytab /opt/kafka.keytab <match *> @type stdout output_type json 2018-10-21 12:32:38 +0800 [info]: initialized kafka producer: kafka 2018-10-21 12:32:38 +0800 [info]: following tail of /mnt/old/tx-failover.log 2018-10-21 12:32:49 +0800 [info]: detected rotation of /mnt/old/tx-failover.log; waiting 5 seconds 2018-10-21 12:32:49 +0800 [info]: following tail of /mnt/old/tx-failover.log 2018-10-21 12:32:49 +0800 fluent.info: {"message":"detected rotation of /mnt/old/tx-failover.log; waiting 5 seconds"} 2018-10-21 12:32:49 +0800 fluent.info: {"message":"following tail of /mnt/old/tx-failover.log"} 2018-10-21 12:33:38 +0800 [trace]: message will send to log4j with partition_key: , partition: , message_key: and value: {"widget":{"debug":"on","window":{"title":"Sample Konfabulator Widget","name":"main_window","width":500,"height":500},"image":{"src":"Images/Sun.png","name":"sun1","hOffset":250,"vOffset":250,"alignment":"center"},"text":{"data":"Click Here","size":36,"style":"bold","name":"text1","hOffset":250,"vOffset":100,"alignment":"center","onMouseUp":"sun1.opacity = (sun1.opacity / 100) 90;"}}}. 2018-10-21 12:33:38 +0800 [trace]: message will send to log4j with partition_key: , partition: , message_key: and value: {"widget":{"debug":"on","window":{"title":"Sample Konfabulator Widget","name":"main_window","width":500,"height":500},"image":{"src":"Images/Sun.png","name":"sun1","hOffset":250,"vOffset":250,"alignment":"center"},"text":{"data":"Click Here","size":36,"style":"bold","name":"text1","hOffset":250,"vOffset":100,"alignment":"center","onMouseUp":"sun1.opacity = (sun1.opacity / 100) 90;"}}}. 2018-10-21 12:33:38 +0800 [trace]: message will send to log4j with partition_key: , partition: , message_key: and value: {"widget":{"debug":"on","window":{"title":"Sample Konfabulator Widget","name":"main_window","width":500,"height":500},"image":{"src":"Images/Sun.png","name":"sun1","hOffset":250,"vOffset":250,"alignment":"center"},"text":{"data":"Click Here","size":36,"style":"bold","name":"text1","hOffset":250,"vOffset":100,"alignment":"center","onMouseUp":"sun1.opacity = (sun1.opacity / 100) 90;"}}}. 2018-10-21 12:33:38 +0800 [debug]: 3 messages send. 2018-10-21 12:33:38 +0800 [info]: New topics added to target list: log4j 2018-10-21 12:33:38 +0800 [info]: Fetching cluster metadata from kafka://192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: Opening connection to 192.168.5.129:9092 with client id kafka... 2018-10-21 12:33:38 +0800 fluent.trace: {"message":"message will send to log4j with partition_key: , partition: , message_key: and value: {\"widget\":{\"debug\":\"on\",\"window\":{\"title\":\"Sample Konfabulator Widget\",\"name\":\"main_window\",\"width\":500,\"height\":500},\"image\":{\"src\":\"Images/Sun.png\",\"name\":\"sun1\",\"hOffset\":250,\"vOffset\":250,\"alignment\":\"center\"},\"text\":{\"data\":\"Click Here\",\"size\":36,\"style\":\"bold\",\"name\":\"text1\",\"hOffset\":250,\"vOffset\":100,\"alignment\":\"center\",\"onMouseUp\":\"sun1.opacity = (sun1.opacity / 100) 90;\"}}}."} 2018-10-21 12:33:38 +0800 fluent.trace: {"message":"message will send to log4j with partition_key: , partition: , message_key: and value: {\"widget\":{\"debug\":\"on\",\"window\":{\"title\":\"Sample Konfabulator Widget\",\"name\":\"main_window\",\"width\":500,\"height\":500},\"image\":{\"src\":\"Images/Sun.png\",\"name\":\"sun1\",\"hOffset\":250,\"vOffset\":250,\"alignment\":\"center\"},\"text\":{\"data\":\"Click Here\",\"size\":36,\"style\":\"bold\",\"name\":\"text1\",\"hOffset\":250,\"vOffset\":100,\"alignment\":\"center\",\"onMouseUp\":\"sun1.opacity = (sun1.opacity / 100) 90;\"}}}."} 2018-10-21 12:33:38 +0800 fluent.trace: {"message":"message will send to log4j with partition_key: , partition: , message_key: and value: {\"widget\":{\"debug\":\"on\",\"window\":{\"title\":\"Sample Konfabulator Widget\",\"name\":\"main_window\",\"width\":500,\"height\":500},\"image\":{\"src\":\"Images/Sun.png\",\"name\":\"sun1\",\"hOffset\":250,\"vOffset\":250,\"alignment\":\"center\"},\"text\":{\"data\":\"Click Here\",\"size\":36,\"style\":\"bold\",\"name\":\"text1\",\"hOffset\":250,\"vOffset\":100,\"alignment\":\"center\",\"onMouseUp\":\"sun1.opacity = (sun1.opacity / 100) * 90;\"}}}."} 2018-10-21 12:33:38 +0800 fluent.debug: {"message":"3 messages send."} 2018-10-21 12:33:38 +0800 fluent.info: {"message":"New topics added to target list: log4j"} 2018-10-21 12:33:38 +0800 fluent.info: {"message":"Fetching cluster metadata from kafka://192.168.5.129:9092"} 2018-10-21 12:33:38 +0800 fluent.debug: {"message":"Opening connection to 192.168.5.129:9092 with client id kafka..."} 2018-10-21 12:33:38 +0800 [debug]: Sending sasl_handshake API request 1 to 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: Waiting for response 1 from 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: Received response 1 from 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: GSSAPI: Initializing context with 192.168.5.129:9092, principal kafka/cipkafka1t.testesunbank.com.tw@KAFKA.COM 2018-10-21 12:33:38 +0800 [debug]: Sending topic_metadata API request 2 to 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: Waiting for response 2 from 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: Closing socket to 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [debug]: Closing socket to 192.168.5.129:9092 2018-10-21 12:33:38 +0800 [error]: Failed to fetch metadata from kafka://192.168.5.129:9092: Connection error EOFError: end of file reached 2018-10-21 12:33:38 +0800 [warn]: Send exception occurred: Could not connect to any of the seed brokers:

Following is my local gems ` LOCAL GEMS

actionmailer (4.2.8) actionpack (4.2.8) actionview (4.2.8) activejob (4.2.8) activemodel (4.2.8) activerecord (4.2.8) activesupport (4.2.8) addressable (2.5.2, 2.5.1) arel (6.0.4) aws-sdk (2.10.45) aws-sdk-core (2.10.45) aws-sdk-resources (2.10.45) aws-sigv4 (1.0.2) base91 (0.0.1) bigdecimal (default: 1.2.4) bson (4.1.1) builder (3.2.3) bundler (1.14.5) celluloid (0.15.2) cool.io (1.5.1) diff-lcs (1.3) digest-crc (0.4.1) draper (1.4.0) elasticsearch (5.0.5) elasticsearch-api (5.0.5) elasticsearch-transport (5.0.5) erubis (2.7.0) excon (0.62.0) faraday (0.13.1) ffi (1.9.25) fluent-logger (0.7.1) fluent-mixin-plaintextformatter (0.2.6) fluent-plugin-elasticsearch (1.17.2) fluent-plugin-genhashvalue (0.04) fluent-plugin-kafka (0.8.0) fluent-plugin-mongo (0.8.1) fluent-plugin-rewrite-tag-filter (1.5.6) fluent-plugin-s3 (0.8.5) fluent-plugin-scribe (0.10.14) fluent-plugin-td (0.10.29) fluent-plugin-td-monitoring (0.2.3) fluent-plugin-webhdfs (0.7.1) fluentd (0.12.40) fluentd-ui (0.4.4) font-awesome-rails (4.7.0.1) globalid (0.4.0) gssapi (1.2.0) haml (4.0.7) haml-rails (0.5.3) hike (1.2.3) hirb (0.7.3) http_parser.rb (0.6.0) httpclient (2.8.2.4) i18n (0.8.1) io-console (default: 0.4.3) ipaddress (0.8.3) jbuilder (2.6.3) jmespath (1.3.1) jquery-rails (3.1.4) json (default: 1.8.1) kramdown (1.13.2) kramdown-haml (0.0.3) loofah (2.0.3) ltsv (0.1.0) mail (2.6.4) mime-types (3.1) mime-types-data (3.2016.0521) mini_portile2 (2.3.0, 2.1.0) minitest (5.10.1, default: 4.7.5) mixlib-cli (1.7.0) mixlib-config (2.2.4) mixlib-log (1.7.1) mixlib-shellout (2.2.7) mongo (2.2.7) msgpack (1.1.0) multi_json (1.12.1) multipart-post (2.0.0) nokogiri (1.8.1) ohai (6.20.0) oj (2.18.1) parallel (1.8.0) psych (default: 2.0.5) public_suffix (3.0.0, 2.0.5) puma (3.8.2) rack (1.6.5) rack-test (0.6.3) rails (4.2.8) rails-deprecated_sanitizer (1.0.3) rails-dom-testing (1.0.8) rails-html-sanitizer (1.0.3) railties (4.2.8) rake (default: 10.1.0) rdoc (default: 4.1.0) request_store (1.3.2) ruby-kafka (0.7.3) ruby-progressbar (1.8.3) rubyzip (1.2.1, 1.1.7) sass (3.2.19) sass-rails (4.0.5) settingslogic (2.0.9) sigdump (0.2.4) sprockets (2.12.4) sprockets-rails (2.3.3) string-scrub (0.0.5) sucker_punch (1.0.5) systemu (2.5.2) td (0.15.2) td-client (0.8.85) td-logger (0.3.27) test-unit (default: 2.1.10.0) thor (0.19.4) thread_safe (0.3.6) thrift (0.8.0) tilt (1.4.1) timers (1.1.0) tzinfo (1.2.3) tzinfo-data (1.2017.2) uuidtools (2.1.5) webhdfs (0.8.0) yajl-ruby (1.3.0) zip-zip (0.3)`

Please help me. Thanks los.

repeatedly commented 5 years ago

Does anyone know how to fix this problem? kerberos feature is contributed by user and it depends on ruby-kafka's kerberos support. So kerberos support should work on basic stetup but I'm not sure what is wrong.

openchung commented 5 years ago

I have asked the question in ruby-kafka's discussion issues. #670 .We have an urgent need for help.

mihir2402 commented 5 years ago

@openchung I am facing the same issue for Kerberos with GSSAPI. Were you/anyone able to fix it?

dborysenko commented 4 years ago

Same exact issue in my environment. Is there success story of working GSSAPI?

frencopei commented 4 years ago

same issue in my environment with keytab and principal but not krb5.conf.

i think the config is not completed ,you wont find KDC server addr without krb5.conf. any successful story?

rymonroe commented 4 years ago

I have same issue, I've confirmed keytab/principle with Kinit. Anyone get this working?

xidiandb commented 3 years ago

i think the config is not completed ,you wont find KDC server addr without krb5.conf. any successful story?

Solved?

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

victorpalmeida commented 2 years ago

I have the same problem. I've provided principal and keytab. Also I can validate using kinit that the kyetab and principal are valid, it is returning a Kerberos ticket. Any news or successful story?

swananddh commented 2 years ago

@ashie could you please update if this issue is resolved? or if any workaround available for above issue then that will be helpful.

Thor77 commented 2 years ago

As long as the issue in the ruby library is not solved (https://github.com/zendesk/ruby-kafka/issues/670) this one can't be solved either :/

dborysenko commented 2 years ago

I'd suggest you to use rdkafka. It works just fine with Fluentd kerberos authN.