search

tchiotludo / akhq

Kafka GUI for Apache Kafka to manage topics, topics data, consumers group, schema registry, connect and more...
https://akhq.io/
Apache License 2.0
3.34k stars 646 forks source link

KafkaHQ hang when live tailing with multiple browser #187

Closed ankon closed 2 years ago

ankon commented 4 years ago

I can reproduce this reliably, but haven't yet had time to look into this further.

  1. Open a browser window, open KafkaHQ in it, and start live tailing a topic
  2. Try to open another window to KafkaHQ

In my case the 2nd window will hang without ever rendering anything of KafkaHQ. As soon as a I stop the live tailing in the first window the second window will load.

Possibly relevant stuff:

jorgheymans commented 4 years ago

confirming this, multiple tabs are ok though ...

HungUnicorn commented 4 years ago

I'd suggest to test with a local kafka. it could be related to the network connecting to kafka, or?

ankon commented 4 years ago

In our case: Pretty sure the problem isn't Kafka or the network to it:

  1. Kafka is MSK, and multiple services (~40+) are working of that same kafka at that time
  2. The network infrastructure itself is a AWS EKS cluster, where both the services and KafkaHQ are in
  3. Being able to talk in the first tab to KafkaHQ and seeing messages in the browser means the network from me to KafkaHQ to MSK is working
tchiotludo commented 4 years ago

I'm also almost sure that I must break the event loop of micronaut by one way

svaranasi-corporate commented 4 years ago

We're also experiencing the same issue with Live Tail. It absolutely kills the application for everyone else. Even tried increasing Xmx to 6g but it isn't helping.

Any other JVM options that might help ease this situation?

Is it possible to disable the live-tail feature?

tchiotludo commented 4 years ago

Hi there.

I never found a way to reproduce the issue .... I made a small change that can lead to better result but not really sure about that.

Can you try with the dev version please ?

svaranasi-corporate commented 4 years ago

Hi there.

I never found a way to reproduce the issue .... I made a small change that can lead to better result but not really sure about that.

Can you try with the dev version please ?

Just tried with the dev version. The UI is slightly different with selecting a topic, but the result is the same. It freezes up the website for all other users.

A couple of questions/concerns:

Thank you for being responsive. It is appreciated.

tchiotludo commented 4 years ago

To be honest, my way is more to understand & fix the bug than to disable the feature :smile: Maybe some debug log could help :

Maybe I can found some information looking at that :pray:

mwesox commented 4 years ago

We have the same issue ... running AWS MSK and AKHQ on Kubernetes (Openshift) ... small difference ... it hangs even with only one tab open

tchiotludo commented 4 years ago

@mwesox Maybe you can provide some log & metrics to help me to understand ?

mwesox commented 4 years ago 

2020-07-07 20:12:45,594 ERROR pGroup-1-3 o.a.c.ErrorController      Error for List topics
--
  | java.lang.RuntimeException: Error for List topics
  | at org.akhq.utils.Logger.call(Logger.java:26)
  | at org.akhq.modules.AbstractKafkaWrapper.listTopics(AbstractKafkaWrapper.java:52)
  | at org.akhq.modules.$KafkaWrapperRequestScopeDefinition$$exec2.invokeInternal(Unknown Source)
  | at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
  | at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:60)
  | at org.akhq.modules.$KafkaWrapperRequestScopeDefinition$Intercepted.listTopics(Unknown Source)
  | at org.akhq.repositories.TopicRepository.all(TopicRepository.java:71)
  | at org.akhq.repositories.TopicRepository.list(TopicRepository.java:63)
  | at org.akhq.controllers.TopicController.list(TopicController.java:121)
  | at org.akhq.controllers.$TopicControllerDefinition$$exec1.invokeInternal(Unknown Source)
  | at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
  | at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:2995)
  | at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:286)
  | at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:122)
  | at io.micronaut.http.server.netty.RoutingInBoundHandler.lambda$buildResultEmitter$16(RoutingInBoundHandler.java:1474)
  | at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:71)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchMap.subscribeActual(FlowableSwitchMap.java:49)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.reactivex.internal.operators.flowable.FlowableDefer.subscribeActual(FlowableDefer.java:42)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FlowableSwitchIfEmpty.java:71)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drainLoop(FlowableFlatMap.java:426)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drain(FlowableFlatMap.java:366)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onComplete(FlowableFlatMap.java:338)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.maybe.MaybeToFlowable$MaybeToFlowableSubscriber.onComplete(MaybeToFlowable.java:80)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybeObserver.onComplete(RxInstrumentedMaybeObserver.java:96)
  | at io.reactivex.internal.operators.maybe.MaybeDoOnEvent$DoOnEventMaybeObserver.onComplete(MaybeDoOnEvent.java:115)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybeObserver.onComplete(RxInstrumentedMaybeObserver.java:96)
  | at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onComplete(FlowableElementAtMaybe.java:102)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drainLoop(FlowableFlatMap.java:426)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drain(FlowableFlatMap.java:366)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onComplete(FlowableFlatMap.java:338)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable$IteratorSubscription.slowPath(FlowableFromIterable.java:255)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable$BaseRangeSubscription.request(FlowableFromIterable.java:124)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onSubscribe(FlowableFlatMap.java:117)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onSubscribe(RxInstrumentedSubscriber.java:54)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribe(FlowableFromIterable.java:69)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribeActual(FlowableFromIterable.java:47)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybe.subscribeActual(RxInstrumentedMaybe.java:53)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.reactivex.internal.operators.maybe.MaybeDoOnEvent.subscribeActual(MaybeDoOnEvent.java:39)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybe.subscribeActual(RxInstrumentedMaybe.java:53)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.reactivex.internal.operators.maybe.MaybeToFlowable.subscribeActual(MaybeToFlowable.java:45)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14868)
  | at io.micronaut.configuration.metrics.binder.web.WebMetricsPublisher.subscribe(WebMetricsPublisher.java:153)
  | at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchMap.subscribeActual(FlowableSwitchMap.java:49)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14868)
  | at io.micronaut.http.context.ServerRequestTracingPublisher.lambda$subscribe$0(ServerRequestTracingPublisher.java:52)
  | at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:68)
  | at io.micronaut.http.context.ServerRequestTracingPublisher.subscribe(ServerRequestTracingPublisher.java:52)
  | at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82)
  | at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:288)
  | at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:253)
  | at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:79)
  | at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:144)
  | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  | at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedRunnable.run(InvocationInstrumenterWrappedRunnable.java:48)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | Caused by: org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.
  | at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:477)
  | at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:467)
  | at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
  | at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)
  | at org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:208)
  | at org.apache.kafka.clients.admin.Admin.create(Admin.java:59)
  | at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:39)
  | at org.akhq.modules.KafkaModule.getAdminClient(KafkaModule.java:100)
  | at org.akhq.modules.AbstractKafkaWrapper.lambda$listTopics$1(AbstractKafkaWrapper.java:53)
  | at org.akhq.utils.Logger.call(Logger.java:19)
  | ... 122 common frames omitted
  | 2020-07-07 20:12:51,810 ERROR pGroup-1-3 o.a.c.ErrorController      Error for List topics
  | java.lang.RuntimeException: Error for List topics
  | at org.akhq.utils.Logger.call(Logger.java:26)
  | at org.akhq.modules.AbstractKafkaWrapper.listTopics(AbstractKafkaWrapper.java:52)
  | at org.akhq.modules.$KafkaWrapperRequestScopeDefinition$$exec2.invokeInternal(Unknown Source)
  | at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
  | at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:60)
  | at org.akhq.modules.$KafkaWrapperRequestScopeDefinition$Intercepted.listTopics(Unknown Source)
  | at org.akhq.repositories.TopicRepository.all(TopicRepository.java:71)
  | at org.akhq.repositories.TopicRepository.list(TopicRepository.java:63)
  | at org.akhq.controllers.TopicController.list(TopicController.java:121)
  | at org.akhq.controllers.$TopicControllerDefinition$$exec1.invokeInternal(Unknown Source)
  | at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
  | at io.micronaut.context.DefaultBeanContext$BeanExecutionHandle.invoke(DefaultBeanContext.java:2995)
  | at io.micronaut.web.router.AbstractRouteMatch.execute(AbstractRouteMatch.java:286)
  | at io.micronaut.web.router.RouteMatch.execute(RouteMatch.java:122)
  | at io.micronaut.http.server.netty.RoutingInBoundHandler.lambda$buildResultEmitter$16(RoutingInBoundHandler.java:1474)
  | at io.reactivex.internal.operators.flowable.FlowableCreate.subscribeActual(FlowableCreate.java:71)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchMap.subscribeActual(FlowableSwitchMap.java:49)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.reactivex.internal.operators.flowable.FlowableDefer.subscribeActual(FlowableDefer.java:42)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FlowableSwitchIfEmpty.java:71)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drainLoop(FlowableFlatMap.java:426)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drain(FlowableFlatMap.java:366)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onComplete(FlowableFlatMap.java:338)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.maybe.MaybeToFlowable$MaybeToFlowableSubscriber.onComplete(MaybeToFlowable.java:80)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybeObserver.onComplete(RxInstrumentedMaybeObserver.java:96)
  | at io.reactivex.internal.operators.maybe.MaybeDoOnEvent$DoOnEventMaybeObserver.onComplete(MaybeDoOnEvent.java:115)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybeObserver.onComplete(RxInstrumentedMaybeObserver.java:96)
  | at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onComplete(FlowableElementAtMaybe.java:102)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drainLoop(FlowableFlatMap.java:426)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.drain(FlowableFlatMap.java:366)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onComplete(FlowableFlatMap.java:338)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onComplete(RxInstrumentedSubscriber.java:97)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable$IteratorSubscription.slowPath(FlowableFromIterable.java:255)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable$BaseRangeSubscription.request(FlowableFromIterable.java:124)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap$MergeSubscriber.onSubscribe(FlowableFlatMap.java:117)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedSubscriber.onSubscribe(RxInstrumentedSubscriber.java:54)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribe(FlowableFromIterable.java:69)
  | at io.reactivex.internal.operators.flowable.FlowableFromIterable.subscribeActual(FlowableFromIterable.java:47)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybe.subscribeActual(RxInstrumentedMaybe.java:53)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.reactivex.internal.operators.maybe.MaybeDoOnEvent.subscribeActual(MaybeDoOnEvent.java:39)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedMaybe.subscribeActual(RxInstrumentedMaybe.java:53)
  | at io.reactivex.Maybe.subscribe(Maybe.java:4290)
  | at io.reactivex.internal.operators.maybe.MaybeToFlowable.subscribeActual(MaybeToFlowable.java:45)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableFlatMap.subscribeActual(FlowableFlatMap.java:53)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchIfEmpty.subscribeActual(FlowableSwitchIfEmpty.java:32)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14868)
  | at io.micronaut.configuration.metrics.binder.web.WebMetricsPublisher.subscribe(WebMetricsPublisher.java:153)
  | at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableSwitchMap.subscribeActual(FlowableSwitchMap.java:49)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.internal.operators.flowable.FlowableMap.subscribeActual(FlowableMap.java:37)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.micronaut.reactive.rxjava2.RxInstrumentedFlowable.subscribeActual(RxInstrumentedFlowable.java:58)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14868)
  | at io.micronaut.http.context.ServerRequestTracingPublisher.lambda$subscribe$0(ServerRequestTracingPublisher.java:52)
  | at io.micronaut.http.context.ServerRequestContext.with(ServerRequestContext.java:68)
  | at io.micronaut.http.context.ServerRequestTracingPublisher.subscribe(ServerRequestTracingPublisher.java:52)
  | at io.reactivex.internal.operators.flowable.FlowableFromPublisher.subscribeActual(FlowableFromPublisher.java:29)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14918)
  | at io.reactivex.Flowable.subscribe(Flowable.java:14865)
  | at io.reactivex.internal.operators.flowable.FlowableSubscribeOn$SubscribeOnSubscriber.run(FlowableSubscribeOn.java:82)
  | at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:288)
  | at io.reactivex.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:253)
  | at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:79)
  | at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:144)
  | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  | at io.micronaut.scheduling.instrument.InvocationInstrumenterWrappedRunnable.run(InvocationInstrumenterWrappedRunnable.java:48)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | Caused by: org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.
  | at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:477)
  | at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:467)
  | at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:108)
  | at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:142)
  | at org.apache.kafka.clients.admin.AdminClientConfig.<init>(AdminClientConfig.java:208)
  | at org.apache.kafka.clients.admin.Admin.create(Admin.java:59)
  | at org.apache.kafka.clients.admin.AdminClient.create(AdminClient.java:39)
  | at org.akhq.modules.KafkaModule.getAdminClient(KafkaModule.java:100)
  | at org.akhq.modules.AbstractKafkaWrapper.lambda$listTopics$1(AbstractKafkaWrapper.java:53)
  | at org.akhq.utils.Logger.call(Logger.java:19)
  | ... 122 common frames omitted
  | 2020-07-07 20:12:56,512 ERROR pGroup-1-2 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | 2020-07-07 20:12:57,004 ERROR pGroup-1-3 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | 2020-07-07 20:12:57,234 ERROR pGroup-1-2 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | 2020-07-07 20:12:57,988 ERROR pGroup-1-2 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | 2020-07-07 20:12:58,047 ERROR pGroup-1-3 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | 2020-07-07 20:13:01,705 ERROR pGroup-1-3 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
  | 2020-07-07 20:13:17,353 ERROR pGroup-1-3 o.a.c.ErrorController      Connection reset by peer
  | java.io.IOException: Connection reset by peer
  | at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
  | at java.base/sun.nio.ch.SocketDispatcher.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.IOUtil.read(Unknown Source)
  | at java.base/sun.nio.ch.SocketChannelImpl.read(Unknown Source)
  | at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
  | at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
  | at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
  | at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
  | at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
  | at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
  | at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
  | at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
  | at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
  | at java.base/java.lang.Thread.run(Unknown Source)
mwesox commented 4 years ago

Metrics

{"names":["executor","executor.active","executor.completed","executor.pool.size","executor.queue.remaining","executor.queued","http.server.requests","jvm.buffer.count","jvm.buffer.memory.used","jvm.buffer.total.capacity","jvm.classes.loaded","jvm.classes.unloaded","jvm.gc.live.data.size","jvm.gc.max.data.size","jvm.gc.memory.allocated","jvm.gc.memory.promoted","jvm.gc.pause","jvm.memory.committed","jvm.memory.max","jvm.memory.used","jvm.threads.daemon","jvm.threads.live","jvm.threads.peak","jvm.threads.states","logback.events","process.cpu.usage","process.files.max","process.files.open","process.start.time","process.uptime","system.cpu.count","system.cpu.usage","system.load.average.1m"]}
ankon commented 4 years ago

So if I understand correctly this is supposedly fixed with 0.15.0?

tchiotludo commented 4 years ago

Not really, it was a try that do better than worst but don't seems to work. I was thinking about it, I've the feeling that is must be for all people that use a session (auth). Maybe the session is locked on micronaut.

@mwesox & @ankon is this your case (authentification enabled) ?

mwesox commented 4 years ago

@tchiotludo : You mean authentication to KafkaHQ? It is disabled in our case.

tchiotludo commented 4 years ago

arf thanks @mwesox , wrong feeling ...

and I just see that I forgot a response :beach_umbrella:

I was wrong on the metrics url :

maybe this can help ...

mwesox commented 4 years ago

@tchiotludo : Maybe try better to reproduce it yourself ... seems like all users running KafkaHQ on kubernetes have the issue .. so maybe try to setup minishift or so and run it there ? Next step maybe run it on AWS EKS ?

jorgheymans commented 4 years ago

We have the issue, not running on k8s just standalone jar, with ldap Auth.

Op di 4 aug. 2020 18:19 schreef Matthias notifications@github.com:

@tchiotludo https://github.com/tchiotludo : Maybe try better to reproduce it yourself ... seems like all users running KafkaHQ on kubernetes have the issue .. so maybe try to setup minishift or so and run it there ? Next step maybe run it on AWS EKS ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/tchiotludo/akhq/issues/187#issuecomment-668691918, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABPKAGAMWFVED3H4Y5F4K3R7AYHNANCNFSM4KHFVOSQ .

tchiotludo commented 4 years ago

To be honest, I'll try to reproduce myself and I'm really not able to reproduce it ! I'll try with docker / k8s and standalone, it don't seems to be the akhq env that seems to cause the issue. I don't have any kafka cluster on AWS or any AWS account, so can't test it.

The main feeling I have is based on authentication (mostly session locking) but don't seems to be the case.

So clearly I have no clue and no reproduction, If no one send me some logs / metrics / configuration, I will not be able to find the issue.

dinegri commented 3 years ago

Same issue for me

I am running akhq dev docker version in ECS behind a reverse proxy using nginx

I will try to getting some logs

tchiotludo commented 3 years ago

@ankon @dinegri @mwesox @jorgheymans @svaranasi-traderev .

I'm not really sure since I can't reproduce the issue even trying to add some Thread.sleep everywhere !

I've made some change about that here : https://github.com/tchiotludo/akhq/commit/60a33bb5965a340c345b323c6e9d690bf270de26

Can you try with dev image please ?

dajudge commented 3 years ago

Can we maybe at least have a config option to disable the live tailing feature? Would you be interested in a PR for that?

tchiotludo commented 3 years ago

@dajudge why disabled ? it's a really issue for you now ?

dajudge commented 3 years ago

as using the live tailing feature seems to block further HTTP communication this leads to readiness-/liveness-probes failing for the container in k8s - which in turn makes our gitops tooling unhappy and triggers alerting. tbh we'd be better off to just disable the feature entirely so no user gets tempted to use it and use some alternative like kafkacat or so...

tchiotludo commented 3 years ago

@dajudge : https://github.com/micronaut-projects/micronaut-core/discussions/5080

Hope a response to clearly fix that :smile:

dajudge commented 3 years ago

@tchiotludo Thanks for looking into it! Let me know if we can help!

voriol commented 3 years ago

I had the same problem:

  1. Open a browser window, open KafkaHQ in it, and start live tailing a topic
  2. Try to open another window to KafkaHQ, but it never "opens"

My architecture is the following: AWS +Kops +EC2 +kubernetes

When I checked kuvernetes events, I noticed how I had "redinessprobe" problems, looking for the possible causes I found the following kubernetes "bug":

https://github.com/kubernetes/kubernetes/issues/89898

So I provided one of the workarounds they proposed, and changed the httpGet with to a simple exec running curl. This change implied updating the docker image, installing curl, and the helm chart, changing the redinessprobe.

Dockerfile:

FROM tchiotludo/akhq:0.17.0

USER root

RUN apt-get update && apt-get install -y curl \
    && rm -rf /var/lib/apt/lists/*

USER akhq

deployment.yaml: 

          readinessProbe:
            exec:
              command:
                - "curl"
                - "-s"
                - "http://localhost:8081{{ .Values.readinessProbe.prefix | default "" }}/health"

With these changes I was able to open several "trails" in several tabs.

On the other hand, and taking advantage of the fact that I had to clone and modify the helm chart, I took the opportunity to customize some parameters of the jvm, since the performance of the application was not as expected, or it was not like that of the on-premises version (without kubernetes)

JVM:

  -Xms2048m
  -Xmx2048m

kubernetes limits & requests

resources:
  limits:
    memory: 2.5Gi
  requests:
    cpu: 1
    memory: 2.5Gi

With these changes I was able to enjoy a fully functional installation on kubernetes

@tchiotludo If you think that all this can be of help to you, I can make you a pull request with all the changes that worked for me.

haidaraM commented 3 years ago

Just for information: same issue on MSK + AWS ALB + ECS on Fargate in our case. We also use Cognito in front of our ALB for Authentication.

tchiotludo commented 3 years ago

here is the final result : https://github.com/micronaut-projects/micronaut-core/issues/5123. It's a bug in micronaut, will wait for the fix and if not coming, will use the workaround

ebrard commented 3 years ago

Looks like the fix has been merged in the Micronaut project, not sure tho if this is available in 2.5.5.

tchiotludo commented 3 years ago

@ebrard it's available on the micronaut 2.5.5 and available on dev, you can test please ? :+1:

ebrard commented 3 years ago

I've just tried against a production cluster and it seems indeed to work better. I've tried only with one active tail view, but this was already slowing down every other calls before.