Kafka not visible and documents not processed

Following the userguide, the extraction procedure does not start. Has it anything to do with kafka listeners?

kafka_1 | [2019-04-04 17:14:59,223] INFO KafkaConfig values: kafka_1 | advertised.host.name = kafka kafka_1 | advertised.listeners = null kafka_1 | advertised.port = 9092

Logstash first outputs several exceptions as follows: logstash_1 | [2019-04-04T17:15:09,203][INFO ][org.apache.kafka.common.utils.AppInfoParser] Kafka version : 0.10.0.1 logstash_1 | [2019-04-04T17:15:09,203][INFO ][org.apache.kafka.common.utils.AppInfoParser] Kafka commitId : a7a17cdec9eaa6c5 logstash_1 | [2019-04-04T17:15:09,204][WARN ][org.apache.kafka.common.utils.AppInfoParser] Error registering AppInfo mbean logstash_1 | javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=logstash-3 logstash_1 | at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437) ~[?:1.8.0_151] logstash_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898) ~[?:1.8.0_151] logstash_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966) ~[?:1.8.0_151] logstash_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900) ~[?:1.8.0_151] logstash_1 | at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324) ~[?:1.8.0_151] logstash_1 | at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) ~[?:1.8.0_151] logstash_1 | at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:58) [kafka-clients-0.10.0.1.jar:?] logstash_1 | at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:694) [kafka-clients-0.10.0.1.jar:?] logstash_1 | at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:587) [kafka-clients-0.10.0.1.jar:?] logstash_1 | at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:569) [kafka-clients-0.10.0.1.jar:?] logstash_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_151] logstash_1 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:1.8.0_151] logstash_1 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:1.8.0_151] logstash_1 | at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [?:1.8.0_151] logstash_1 | at org.jruby.javasupport.JavaConstructor.newInstanceDirect(JavaConstructor.java:277) [?:?] logstash_1 | at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:83) [?:?] logstash_1 | at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:174) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:336) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:179) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:183) [?:?] logstash_1 | at org.jruby.java.proxies.ConcreteJavaProxy$InitializeMethod.call(ConcreteJavaProxy.java:56) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:177) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:183) [?:?] logstash_1 | at org.jruby.RubyClass.newInstance(RubyClass.java:858) [?:?] logstash_1 | at org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrNBlock.call(JavaMethod.java:297) [?:?] logstash_1 | at org.jruby.java.proxies.ConcreteJavaProxy$NewMethod.call(ConcreteJavaProxy.java:155) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168) [?:?] logstash_1 | at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) [?:?] logstash_1 | at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221) [?:?] logstash_1 | at org.jruby.ast.RescueNode.interpret(RescueNode.java:116) [?:?] logstash_1 | at org.jruby.ast.BeginNode.interpret(BeginNode.java:83) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:203) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168) [?:?] logstash_1 | at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112) [?:?] logstash_1 | at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206) [?:?] logstash_1 | at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157) [?:?] logstash_1 | at org.jruby.runtime.Block.yield(Block.java:142) [?:?] logstash_1 | at org.jruby.RubyEnumerable$23.call(RubyEnumerable.java:770) [?:?] logstash_1 | at org.jruby.runtime.CallBlock19.yield(CallBlock19.java:90) [?:?] logstash_1 | at org.jruby.runtime.Block.yield(Block.java:142) [?:?] logstash_1 | at org.jruby.RubyFixnum.times(RubyFixnum.java:280) [?:?] logstash_1 | at org.jruby.RubyFixnum$INVOKER$i$0$0$times.call(RubyFixnum$INVOKER$i$0$0$times.gen) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:458) [?:?] logstash_1 | at org.jruby.RubyClass.finvoke(RubyClass.java:545) [?:?] logstash_1 | at org.jruby.runtime.Helpers.invoke(Helpers.java:498) [?:?] logstash_1 | at org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:388) [?:?] logstash_1 | at org.jruby.RubyEnumerator.each(RubyEnumerator.java:299) [?:?] logstash_1 | at org.jruby.RubyEnumerator$INVOKER$i$each.call(RubyEnumerator$INVOKER$i$each.gen) [?:?] logstash_1 | at org.jruby.RubyClass.finvoke(RubyClass.java:524) [?:?] logstash_1 | at org.jruby.runtime.Helpers.invoke(Helpers.java:486) [?:?] logstash_1 | at org.jruby.RubyEnumerable.callEach19(RubyEnumerable.java:104) [?:?] logstash_1 | at org.jruby.RubyEnumerable.collectCommon19(RubyEnumerable.java:762) [?:?] logstash_1 | at org.jruby.RubyEnumerable.map19(RubyEnumerable.java:754) [?:?] logstash_1 | at org.jruby.RubyEnumerable$INVOKER$s$0$0$map19.call(RubyEnumerable$INVOKER$s$0$0$map19.gen) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:143) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154) [?:?] logstash_1 | at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64) [?:?] logstash_1 | at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) [?:?] logstash_1 | at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:203) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168) [?:?] logstash_1 | at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) [?:?] logstash_1 | at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221) [?:?] logstash_1 | at org.jruby.ast.RescueNode.interpret(RescueNode.java:116) [?:?] logstash_1 | at org.jruby.ast.EnsureNode.interpret(EnsureNode.java:96) [?:?] logstash_1 | at org.jruby.ast.BeginNode.interpret(BeginNode.java:83) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) [?:?] logstash_1 | at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182) [?:?] logstash_1 | at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:203) [?:?] logstash_1 | at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:168) [?:?] logstash_1 | at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36) [?:?] logstash_1 | at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105) [?:?] logstash_1 | at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112) [?:?] logstash_1 | at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206) [?:?] logstash_1 | at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:194) [?:?] logstash_1 | at org.jruby.runtime.Interpreted19Block.call(Interpreted19Block.java:125) [?:?] logstash_1 | at org.jruby.runtime.Block.call(Block.java:101) [?:?] logstash_1 | at org.jruby.RubyProc.call(RubyProc.java:300) [?:?] logstash_1 | at org.jruby.RubyProc.call(RubyProc.java:230) [?:?] logstash_1 | at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:103) [?:?] logstash_1 | at java.lang.Thread.run(Thread.java:748)[?:1.8.0_151]``

and aftewards it is giving the following output:

logstash_1 | [2019-04-04T17:15:13,699][WARN ][org.apache.kafka.clients.NetworkClient] Error while fetching metadata with correlation id 80 :

{new_project_out=LEADER_NOT_AVAILABLE}

usc-isi-i2 / dig-etl-engine

Kafka not visible and documents not processed #276