Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

[Abdulsamirkhan]

can you please help me to fix this error, i tried all possible combinations

I am able to able to read topic metadata, produce message and consume message from c# code.

---

Pinot docker image: [apachepinot/pinot:latest]

---

  Command 
  bin/pinot-admin.sh AddTable -tableConfigFile  /tmp/SamirTestTable1.json -schemaFile /tmp/SamirTestSchema.json -controllerHost localhost -controllerPort 9000 -exec

---

Kafka : server.properties : message.max.bytes=2200000000 listeners=PLAINTEXT://localhost:9092,PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092 zookeeper.connect=localhost:2181

Consumer.properties
    bootstrap.servers=localhost:29092
    group.id=test-consumer-group
    message.max.bytes=2200000000
    max.request.size=1200000000
    metadata.max.age.ms=300000
    heartbeat.interval.ms=1000
    request.timeout.ms=160000
    max.partition.fetch.bytes =2200000000
    fetch.metadata.timeout.ms =600000 
    session.timeout.ms = 145000 
    fetch.max.bytes = 2200000000

---

Compose Yaml kafka: image: confluentinc/cp-kafka:5.5.0 depends_on:

• zookeeper ports:

• "29092:29092" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT

  Schema Json file { "schemaName": "SamirTest", "dimensionFieldSpecs": [ {"name": "uuid", "dataType": "STRING"} ], "metricFieldSpecs": [ {"name": "count", "dataType": "INT"} ], "dateTimeFieldSpecs": [ {"name": "SamirTestTime", "dataType": "LONG", "format": "1:MILLISECONDS:EPOCH", "granularity": "1:MILLISECONDS"} ] }

  Table Json file
  { "tableName": "SamirTest_REALTIME", "tableType": "REALTIME", "segmentsConfig": { "timeColumnName": "SamirTestTime", "schemaName": "SamirTest", "replication": "1" }, "tenants": { "broker": "DefaultTenant", "server": "DefaultTenant" }, "metadata": {}, "tableIndexConfig": { "loadMode": "MMAP", "streamConfigs": { "streamType": "kafka", "stream.kafka.consumer.type": "simple", "stream.kafka.topic.name": "samirtestNew", "stream.kafka.broker.list": "localhost:29092", "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder", "stream.kafka.consumer.prop.auto.offset.reset": "smallest", "stream.kafka.consumer.prop.fetch.metadata.timeout.ms": "120000", "stream.kafka.consumer.prop.session.timeout.ms": "60000", "stream.kafka.consumer.prop.request.timeout.ms": "120000", "realtime.segment.flush.threshold.rows": "50000", "realtime.segment.flush.threshold.time": "24h" } } }

---

Code used to get metadata and worked

              var config1 = new ConsumerConfig
              {
                  BootstrapServers = "localhost:29092",
                  GroupId = "metadata-fetcher-group",
                  AutoOffsetReset = AutoOffsetReset.Earliest,
                  EnableAutoCommit = false
              };

              using (var adminClient = new AdminClientBuilder(config).Build())
              {
                  try
                  {
                      // Fetch metadata for all topics
                      var metadata = adminClient.GetMetadata(TimeSpan.FromSeconds(30));

                      foreach (var topic in metadata.Topics)
                      {
                          Console.WriteLine($"Topic: {topic.Topic}");
                          Console.WriteLine($"Partition Count: {topic.Partitions.Count}");
                          foreach (var partition in topic.Partitions)
                          {
                              Console.WriteLine($"Partition: {partition.PartitionId}, Leader: {partition.Leader}, Replicas: {string.Join(",", partition.Replicas)}, ISR: {string.Join(",", partition.InSyncReplicas)}");
                          }
                      }
                  }
                  catch (KafkaException e)
                  {
                      Console.WriteLine($"An error occurred: {e.Error.Reason}");
                  }
              }

---

Error details : 2024-11-04 17:39:19 2024/11/04 12:09:19.104 ERROR [PinotTableIdealStateBuilder] [grizzly-http-server-5] Could not get PartitionGroupMetadata for topic: samirtestNew of table: SamirTest_REALTIME 2024-11-04 17:39:19 org.apache.pinot.spi.stream.TransientConsumerException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:19 at org.apache.pinot.plugin.stream.kafka20.KafkaStreamMetadataProvider.fetchPartitionCount(KafkaStreamMetadataProvider.java:72) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.stream.StreamMetadataProvider.computePartitionGroupMetadata(StreamMetadataProvider.java:76) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.stream.PartitionGroupMetadataFetcher.call(PartitionGroupMetadataFetcher.java:70) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.stream.PartitionGroupMetadataFetcher.call(PartitionGroupMetadataFetcher.java:31) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:58) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.PinotTableIdealStateBuilder.getPartitionGroupMetadataList(PinotTableIdealStateBuilder.java:93) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.realtime.PinotLLCRealtimeSegmentManager.getNewPartitionGroupMetadataList(PinotLLCRealtimeSegmentManager.java:799) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.realtime.PinotLLCRealtimeSegmentManager.setUpNewTable(PinotLLCRealtimeSegmentManager.java:325) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.PinotHelixResourceManager.addTable(PinotHelixResourceManager.java:1746) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.api.resources.TableConfigsRestletResource.addConfig(TableConfigsRestletResource.java:233) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] 2024-11-04 17:39:19 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] 2024-11-04 17:39:19 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] 2024-11-04 17:39:19 at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:190) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at java.base/java.lang.Thread.run(Thread.java:829) [?:?] 2024-11-04 17:39:19 Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:19 2024/11/04 12:09:19.302 ERROR [PinotHelixResourceManager] [grizzly-http-server-5] Caught exception while setting up table: SamirTest_REALTIME, cleaning it up 2024-11-04 17:39:19 java.lang.RuntimeException: org.apache.pinot.spi.stream.TransientConsumerException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.PinotTableIdealStateBuilder.getPartitionGroupMetadataList(PinotTableIdealStateBuilder.java:99) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.realtime.PinotLLCRealtimeSegmentManager.getNewPartitionGroupMetadataList(PinotLLCRealtimeSegmentManager.java:799) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.realtime.PinotLLCRealtimeSegmentManager.setUpNewTable(PinotLLCRealtimeSegmentManager.java:325) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.PinotHelixResourceManager.addTable(PinotHelixResourceManager.java:1746) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.api.resources.TableConfigsRestletResource.addConfig(TableConfigsRestletResource.java:233) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] 2024-11-04 17:39:19 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] 2024-11-04 17:39:19 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] 2024-11-04 17:39:19 at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:190) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at java.base/java.lang.Thread.run(Thread.java:829) [?:?] 2024-11-04 17:39:19 Caused by: org.apache.pinot.spi.stream.TransientConsumerException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:19 at org.apache.pinot.plugin.stream.kafka20.KafkaStreamMetadataProvider.fetchPartitionCount(KafkaStreamMetadataProvider.java:72) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.stream.StreamMetadataProvider.computePartitionGroupMetadata(StreamMetadataProvider.java:76) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.stream.PartitionGroupMetadataFetcher.call(PartitionGroupMetadataFetcher.java:70) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.stream.PartitionGroupMetadataFetcher.call(PartitionGroupMetadataFetcher.java:31) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:58) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 at org.apache.pinot.controller.helix.core.PinotTableIdealStateBuilder.getPartitionGroupMetadataList(PinotTableIdealStateBuilder.java:93) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:19 ... 30 more 2024-11-04 17:39:19 Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:20 2024/11/04 12:09:20.044 ERROR [TableConfigsRestletResource] [grizzly-http-server-5] org.apache.pinot.spi.stream.TransientConsumerException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:20 java.lang.RuntimeException: org.apache.pinot.spi.stream.TransientConsumerException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:20 at org.apache.pinot.controller.helix.core.PinotTableIdealStateBuilder.getPartitionGroupMetadataList(PinotTableIdealStateBuilder.java:99) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.controller.helix.core.realtime.PinotLLCRealtimeSegmentManager.getNewPartitionGroupMetadataList(PinotLLCRealtimeSegmentManager.java:799) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.controller.helix.core.realtime.PinotLLCRealtimeSegmentManager.setUpNewTable(PinotLLCRealtimeSegmentManager.java:325) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.controller.helix.core.PinotHelixResourceManager.addTable(PinotHelixResourceManager.java:1746) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.controller.api.resources.TableConfigsRestletResource.addConfig(TableConfigsRestletResource.java:233) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] 2024-11-04 17:39:20 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?] 2024-11-04 17:39:20 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] 2024-11-04 17:39:20 at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:190) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515) [pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at java.base/java.lang.Thread.run(Thread.java:829) [?:?] 2024-11-04 17:39:20 Caused by: org.apache.pinot.spi.stream.TransientConsumerException: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata 2024-11-04 17:39:20 at org.apache.pinot.plugin.stream.kafka20.KafkaStreamMetadataProvider.fetchPartitionCount(KafkaStreamMetadataProvider.java:72) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.spi.stream.StreamMetadataProvider.computePartitionGroupMetadata(StreamMetadataProvider.java:76) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.spi.stream.PartitionGroupMetadataFetcher.call(PartitionGroupMetadataFetcher.java:70) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.spi.stream.PartitionGroupMetadataFetcher.call(PartitionGroupMetadataFetcher.java:31) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.spi.utils.retry.BaseRetryPolicy.attempt(BaseRetryPolicy.java:58) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 at org.apache.pinot.controller.helix.core.PinotTableIdealStateBuilder.getPartitionGroupMetadataList(PinotTableIdealStateBuilder.java:93) ~[pinot-all-1.3.0-SNAPSHOT-jar-with-dependencies.jar:1.3.0-SNAPSHOT-3057f1ad7b256c0fedbdf293e8758d29f25a22a1] 2024-11-04 17:39:20 ... 30 more 2024-11-04 17:39:20 Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

[Abdulsamirkhan]

@Jackie-Jiang, any help much appreciated

[Jackie-Jiang]

cc @swaminathanmanish @KKcorps to help take a look

[Abdulsamirkhan]

@Jackie-Jiang @swaminathanmanish @KKcorps, Still i am not able to solve this any help much appreciated.

[Abdulsamirkhan]

@Jackie-Jiang @swaminathanmanish @KKcorps, gentle reminder

[Jackie-Jiang]

Can you double check the topic existence and kafka settings? This seems a Kafka issue to me. You may also get more help in the slack troubleshooting channel

[Abdulsamirkhan]

I double checked Topic existence and kafka settings, but not able to point out the issues, let me move to slack if it helps.