search

Cyb3rWard0g / HELK

The Hunting ELK
GNU General Public License v3.0
3.73k stars 675 forks source link

logstash error while fetching metadata with id xxxx #521

Open ssi0202 opened 3 years ago

ssi0202 commented 3 years ago

fresh install on ubuntu the logstash log is full of this, and no data is getting ingested. I have just set up a winlogbeat to ship data from a client machine

below is the full beginning of the *error" part of the logstash log, output is from using the docker follow helk-logstash command

[2020-11-23T14:19:56,088][WARN ][org.apache.kafka.common.utils.AppInfoParser][main] Error registering AppInfo mbean
javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=logstash-0
    at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:436) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) ~[?:?]
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) ~[?:?]
    at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:64) [kafka-clients-2.4.1.jar:?]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:814) [kafka-clients-2.4.1.jar:?]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:666) [kafka-clients-2.4.1.jar:?]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646) [kafka-clients-2.4.1.jar:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:490) [?:?]
    at org.jruby.javasupport.JavaConstructor.newInstanceDirect(JavaConstructor.java:279) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:86) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:175) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:386) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:184) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.proxies.ConcreteJavaProxy$InitializeMethod.call(ConcreteJavaProxy.java:56) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:386) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:184) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass.newInstance(RubyClass.java:894) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrNBlock.call(JavaMethod.java:349) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.proxies.ConcreteJavaProxy$NewMethod.call(ConcreteJavaProxy.java:158) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:375) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:174) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:86) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:73) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_1_dot_0_minus_java.lib.logstash.inputs.kafka.RUBY$block$run$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.1.0-java/lib/logstash/inputs/kafka.rb:233) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:146) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:114) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.yield(Block.java:170) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable$23.call(RubyEnumerable.java:907) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CallBlock19.doYield(CallBlock19.java:111) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:125) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.yieldValues(Block.java:186) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator$2.call(RubyEnumerator.java:409) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockCallback.call(BlockCallback.java:40) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CallBlock.doYield(CallBlock.java:96) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:116) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.yield(Block.java:170) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyFixnum.times(RubyFixnum.java:285) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyInteger$INVOKER$i$0$0$times.call(RubyInteger$INVOKER$i$0$0$times.gen) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass.finvoke(RubyClass.java:505) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Helpers.invoke(Helpers.java:443) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:394) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator.__each__(RubyEnumerator.java:405) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator.each(RubyEnumerator.java:401) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator$INVOKER$i$each.call(RubyEnumerator$INVOKER$i$each.gen) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass.finvoke(RubyClass.java:493) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Helpers.invoke(Helpers.java:431) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable.callEach19(RubyEnumerable.java:121) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable.collectCommon(RubyEnumerable.java:899) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable.map(RubyEnumerable.java:891) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_1_dot_0_minus_java.lib.logstash.inputs.kafka.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.1.0-java/lib/logstash/inputs/kafka.rb:233) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:110) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:253) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:328) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:320) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.call(Block.java:125) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyProc.call(RubyProc.java:274) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105) [jruby-complete-9.2.9.0.jar:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
[2020-11-23T14:19:56,190][WARN ][org.apache.kafka.common.utils.AppInfoParser][main] Error registering AppInfo mbean
javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=logstash-1
    at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:436) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1855) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:955) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:890) ~[?:?]
    at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:320) ~[?:?]
    at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522) ~[?:?]
    at org.apache.kafka.common.utils.AppInfoParser.registerAppInfo(AppInfoParser.java:64) [kafka-clients-2.4.1.jar:?]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:814) [kafka-clients-2.4.1.jar:?]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:666) [kafka-clients-2.4.1.jar:?]
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:646) [kafka-clients-2.4.1.jar:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
    at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [?:?]
    at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [?:?]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:490) [?:?]
    at org.jruby.javasupport.JavaConstructor.newInstanceDirect(JavaConstructor.java:279) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:86) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:175) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:386) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:184) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.proxies.ConcreteJavaProxy$InitializeMethod.call(ConcreteJavaProxy.java:56) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:182) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass.newInstance(RubyClass.java:894) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrOneOrNBlock.call(JavaMethod.java:349) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.java.proxies.ConcreteJavaProxy$NewMethod.call(ConcreteJavaProxy.java:158) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_1_dot_0_minus_java.lib.logstash.inputs.kafka.RUBY$method$create_consumer$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.1.0-java/lib/logstash/inputs/kafka.rb:336) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_1_dot_0_minus_java.lib.logstash.inputs.kafka.RUBY$method$create_consumer$0$__VARARGS__(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.1.0-java/lib/logstash/inputs/kafka.rb) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:84) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_1_dot_0_minus_java.lib.logstash.inputs.kafka.RUBY$block$run$1(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.1.0-java/lib/logstash/inputs/kafka.rb:233) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:146) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:114) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.yield(Block.java:170) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable$23.call(RubyEnumerable.java:907) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CallBlock19.doYield(CallBlock19.java:111) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:125) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.yieldValues(Block.java:186) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator$2.call(RubyEnumerator.java:409) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockCallback.call(BlockCallback.java:40) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CallBlock.doYield(CallBlock.java:96) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:116) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.yield(Block.java:170) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyFixnum.times(RubyFixnum.java:285) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyInteger$INVOKER$i$0$0$times.call(RubyInteger$INVOKER$i$0$0$times.gen) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroBlock.call(JavaMethod.java:555) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass.finvoke(RubyClass.java:505) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Helpers.invoke(Helpers.java:443) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:394) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator.__each__(RubyEnumerator.java:405) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator.each(RubyEnumerator.java:401) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerator$INVOKER$i$each.call(RubyEnumerator$INVOKER$i$each.gen) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyClass.finvoke(RubyClass.java:493) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Helpers.invoke(Helpers.java:431) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable.callEach19(RubyEnumerable.java:121) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable.collectCommon(RubyEnumerable.java:899) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyEnumerable.map(RubyEnumerable.java:891) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_integration_minus_kafka_minus_10_dot_1_dot_0_minus_java.lib.logstash.inputs.kafka.RUBY$method$run$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-integration-kafka-10.1.0-java/lib/logstash/inputs/kafka.rb:233) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:110) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:140) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:253) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$inputworker$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:328) [jruby-complete-9.2.9.0.jar:?]
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start_input$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:320) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:71) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.runtime.Block.call(Block.java:125) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.RubyProc.call(RubyProc.java:274) [jruby-complete-9.2.9.0.jar:?]
    at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105) [jruby-complete-9.2.9.0.jar:?]
    at java.lang.Thread.run(Thread.java:834) [?:?]
[2020-11-23T14:19:56,385][WARN ][org.apache.kafka.clients.NetworkClient][main] [Consumer clientId=logstash-1, groupId=helk_logstash] Error while fetching metadata with correlation id 2 : {winlogbeat=UNKNOWN_TOPIC_OR_PARTITION, SYSMON_JOIN=UNKNOWN_TOPIC_OR_PARTITION, winevent=UNKNOWN_TOPIC_OR_PARTITION, filebeat=UNKNOWN_TOPIC_OR_PARTITION}

[2020-11-23T13:43:03,197][WARN ][org.apache.kafka.clients.NetworkClient][main] [Consumer clientId=logstash-1, groupId=helk_logstash_zeek] Error while fetching metadata with correlation id 19535 : {zeek=UNKNOWN_TOPIC_OR_PARTITION}
[2020-11-23T13:43:03,219][WARN ][org.apache.kafka.clients.NetworkClient][main] [Consumer clientId=logstash-0, groupId=helk_logstash] Error while fetching metadata with correlation id 19534 : {winlogbeat=UNKNOWN_TOPIC_OR_PARTITION, SYSMON_JOIN=UNKNOWN_TOPIC_OR_PARTITION, winevent=UNKNOWN_TOPIC_OR_PARTITION, filebeat=UNKNOWN_TOPIC_OR_PARTITION}
SRJanel commented 3 years ago

Just stumbled upon this issue as well. I managed to quickfix the problem by forcing into logstash’s pipeline configs a different client_id for the two kafka inputs (in 0002-kafka-input.conf and 0006-kafka-zeek-input.conf).

Cyb3rWard0g commented 3 years ago

Hello @ssi0202 and @SRJanel ! Do you still have the same issue?

SRJanel commented 3 years ago

Hello @Cyb3rWard0g My docker containers have been running without any problem since my quickfix. Want me to pull and run again ?

derekking-db commented 2 years ago

I'm seeing this same error. Fresh install. I don't see client_id (i see group_id) in these input files.

Is there a simple fix to this ?

SRJanel commented 2 years ago

Hello @DerekKing001 , try to provide a client_id inside the Kafka input plugin in both files I mentioned above. They should have different values (e.g. client_id => "something1" and client_id => "something2"). Then restart containers and it should work.