search

confluentinc / demo-scene

👾Scripts and samples to support Confluent Demos and Talks. ⚠️Might be rough around the edges ;-) 👉For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/
https://developer.confluent.io
Apache License 2.0
1.5k stars 896 forks source link

build-a-streaming-pipeline's ksqldb and schema-registry meet 2 starter issue. #254

Open Michael2008S opened 1 year ago

Michael2008S commented 1 year ago

Hi confluentors,

I meet 2 starter issues when using docker-compose up -d to run the build-a-streaming-pipeline demo in the current master branch.

image

ksqldb  | [2023-01-30 01:54:56,634] INFO KsqlConfig values: 
ksqldb  |       ksql.access.validator.enable = auto
ksqldb  |       ksql.authorization.cache.expiry.time.secs = 30
ksqldb  |       ksql.authorization.cache.max.entries = 10000
ksqldb  |       ksql.cast.strings.preserve.nulls = true
ksqldb  |       ksql.connect.url = http://kafka-connect-01:8083
ksqldb  |       ksql.connect.worker.config = 
ksqldb  |       ksql.create.or.replace.enabled = true
ksqldb  |       ksql.error.classifier.regex = 
ksqldb  |       ksql.extension.dir = ext
ksqldb  |       ksql.hidden.topics = [^_.*]
ksqldb  |       ksql.insert.into.values.enabled = true
ksqldb  |       ksql.internal.topic.min.insync.replicas = 1
ksqldb  |       ksql.internal.topic.replicas = 1
ksqldb  |       ksql.lambdas.enabled = true
ksqldb  |       ksql.metastore.backup.location = 
ksqldb  |       ksql.metrics.extension = null
ksqldb  |       ksql.metrics.tags.custom = 
ksqldb  |       ksql.nested.error.set.null = true
ksqldb  |       ksql.output.topic.name.prefix = 
ksqldb  |       ksql.persistence.default.format.key = KAFKA
ksqldb  |       ksql.persistence.default.format.value = null
ksqldb  |       ksql.persistence.wrap.single.values = null
ksqldb  |       ksql.persistent.prefix = query_
ksqldb  |       ksql.properties.overrides.denylist = []
ksqldb  |       ksql.pull.queries.enable = true
ksqldb  |       ksql.query.error.max.queue.size = 10
ksqldb  |       ksql.query.persistent.active.limit = 2147483647
ksqldb  |       ksql.query.persistent.max.bytes.buffering.total = -1
ksqldb  |       ksql.query.pull.enable.standby.reads = false
ksqldb  |       ksql.query.pull.interpreter.enabled = true
ksqldb  |       ksql.query.pull.max.allowed.offset.lag = 9223372036854775807
ksqldb  |       ksql.query.pull.max.concurrent.requests = 2147483647
ksqldb  |       ksql.query.pull.max.hourly.bandwidth.megabytes = 2147483647
ksqldb  |       ksql.query.pull.max.qps = 2147483647
ksqldb  |       ksql.query.pull.metrics.enabled = true
ksqldb  |       ksql.query.pull.table.scan.enabled = false
ksqldb  |       ksql.query.pull.thread.pool.size = 100
ksqldb  |       ksql.query.push.scalable.enabled = false
ksqldb  |       ksql.query.push.scalable.interpreter.enabled = true
ksqldb  |       ksql.query.retry.backoff.initial.ms = 15000
ksqldb  |       ksql.query.retry.backoff.max.ms = 900000
ksqldb  |       ksql.query.status.running.threshold.seconds = 300
ksqldb  |       ksql.query.transient.max.bytes.buffering.total = -1
ksqldb  |       ksql.queryanonymizer.cluster_namespace = null
ksqldb  |       ksql.queryanonymizer.logs_enabled = true
ksqldb  |       ksql.readonly.topics = [_confluent.*, __confluent.*, _schemas, __consumer_offsets, __transaction_state, connect-configs, connect-offsets, connect-status, connect-statuses]
ksqldb  |       ksql.schema.registry.url = http://schema-registry:8081
ksqldb  |       ksql.security.extension.class = null
ksqldb  |       ksql.service.id = confluent_rmoff_01
ksqldb  |       ksql.sink.window.change.log.additional.retention = 1000000
ksqldb  |       ksql.streams.shutdown.timeout.ms = 300000
ksqldb  |       ksql.suppress.buffer.size.bytes = -1
ksqldb  |       ksql.suppress.enabled = false
ksqldb  |       ksql.timestamp.throw.on.invalid = false
ksqldb  |       ksql.transient.prefix = transient_
ksqldb  |       ksql.udf.collect.metrics = false
ksqldb  |       ksql.udf.enable.security.manager = true
ksqldb  |       ksql.udfs.enabled = true
ksqldb  |       ksql.variable.substitution.enable = true
ksqldb  |       metric.reporters = []
ksqldb  |       ssl.cipher.suites = null
ksqldb  |       ssl.enabled.protocols = [TLSv1.2, TLSv1.3]
ksqldb  |       ssl.endpoint.identification.algorithm = https
ksqldb  |       ssl.engine.factory.class = null
ksqldb  |       ssl.key.password = null
ksqldb  |       ssl.keymanager.algorithm = SunX509
ksqldb  |       ssl.keystore.certificate.chain = null
ksqldb  |       ssl.keystore.key = null
ksqldb  |       ssl.keystore.location = null
ksqldb  |       ssl.keystore.password = null
ksqldb  |       ssl.keystore.type = JKS
ksqldb  |       ssl.protocol = TLSv1.3
ksqldb  |       ssl.provider = null
ksqldb  |       ssl.secure.random.implementation = null
ksqldb  |       ssl.trustmanager.algorithm = PKIX
ksqldb  |       ssl.truststore.certificates = null
ksqldb  |       ssl.truststore.location = null
ksqldb  |       ssl.truststore.password = null
ksqldb  |       ssl.truststore.type = JKS
ksqldb  |  (io.confluent.ksql.util.KsqlConfig:376)
ksqldb  | log4j:WARN No appenders could be found for logger (org.apache.kafka.connect.json.JsonConverterConfig).
ksqldb  | log4j:WARN Please initialize the log4j system properly.
ksqldb  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
ksqldb  | [2023-01-30 01:55:27,987] ERROR Failed to start KSQL (io.confluent.ksql.rest.server.KsqlServerMain:69)
ksqldb  | java.lang.RuntimeException: Failed to get Kafka cluster information
ksqldb  |       at io.confluent.ksql.services.KafkaClusterUtil.getKafkaClusterId(KafkaClusterUtil.java:107)
ksqldb  |       at io.confluent.ksql.rest.server.KsqlRestApplication.buildApplication(KsqlRestApplication.java:638)
ksqldb  |       at io.confluent.ksql.rest.server.KsqlServerMain.createExecutable(KsqlServerMain.java:155)
ksqldb  |       at io.confluent.ksql.rest.server.KsqlServerMain.main(KsqlServerMain.java:62)
ksqldb  | Caused by: java.util.concurrent.TimeoutException
ksqldb  |       at java.base/java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1886)
ksqldb  |       at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2021)
ksqldb  |       at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:180)
ksqldb  |       at io.confluent.ksql.services.KafkaClusterUtil.getKafkaClusterId(KafkaClusterUtil.java:105)
ksqldb  |       ... 3 more
schema-registry  | [2023-01-30 01:56:37,469] INFO HV000001: Hibernate Validator 6.1.7.Final (org.hibernate.validator.internal.util.Version)
schema-registry  | [2023-01-30 01:56:38,594] INFO Started o.e.j.s.ServletContextHandler@7fc7c4a{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-30 01:56:38,611] INFO Started o.e.j.s.ServletContextHandler@440eaa07{/ws,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-30 01:56:38,714] INFO Started NetworkTrafficServerConnector@2ce6c6ec{HTTP/1.1, (http/1.1)}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector)
schema-registry  | [2023-01-30 01:56:38,714] INFO Started @57225ms (org.eclipse.jetty.server.Server)
schema-registry  | [2023-01-30 01:56:38,714] INFO Server started, listening for requests... (io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain)
schema-registry  | [2023-01-30 01:57:07,637] INFO Registering new schema: subject ratings-value, version null, id null, type null (io.confluent.kafka.schemaregistry.rest.resources.SubjectVersionsResource)
schema-registry  | [2023-01-30 01:57:07,869] INFO Wait to catch up until the offset at 1 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:07,870] INFO Reached offset at 1 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:07,898] INFO Wait to catch up until the offset at 2 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:07,927] INFO Reached offset at 2 (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-30 01:57:08,068] INFO 172.18.0.9 - - [30/Jan/2023:01:57:07 +0000] "POST /subjects/ratings-value/versions HTTP/1.1" 200 8  623 (io.confluent.rest-utils.requests)
schema-registry  | [2023-01-30 02:02:35,266] INFO 172.18.0.1 - - [30/Jan/2023:02:02:35 +0000] "GET / HTTP/1.1" 200 2  18 (io.confluent.rest-utils.requests)
schema-registry  | [2023-01-30 02:02:35,377] ERROR Request Failed with exception  (io.confluent.rest.exceptions.DebuggableExceptionMapper)
schema-registry  | javax.ws.rs.NotFoundException: HTTP 404 Not Found
schema-registry  |      at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:252)
schema-registry  |      at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
schema-registry  |      at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
schema-registry  |      at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
schema-registry  |      at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
schema-registry  |      at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
schema-registry  |      at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
schema-registry  |      at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234)
schema-registry  |      at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680)
schema-registry  |      at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.serviceImpl(ServletContainer.java:386)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:561)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:502)
schema-registry  |      at org.glassfish.jersey.servlet.ServletContainer.doFilter(ServletContainer.java:439)
schema-registry  |      at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
schema-registry  |      at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
schema-registry  |      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry  |      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
schema-registry  |      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
schema-registry  |      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
schema-registry  |      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
schema-registry  |      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350)
schema-registry  |      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
schema-registry  |      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry  |      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
schema-registry  |      at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179)
schema-registry  |      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
schema-registry  |      at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:735)
schema-registry  |      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
schema-registry  |      at org.eclipse.jetty.server.Server.handle(Server.java:516)
schema-registry  |      at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
schema-registry  |      at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
schema-registry  |      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
schema-registry  |      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
schema-registry  |      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
schema-registry  |      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
schema-registry  |      at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
schema-registry  |      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
schema-registry  |      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:882)
schema-registry  |      at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1036)
schema-registry  |      at java.base/java.lang.Thread.run(Thread.java:829)
schema-registry  | [2023-01-30 02:02:35,385] INFO 172.18.0.1 - - [30/Jan/2023:02:02:35 +0000] "GET /favicon.ico HTTP/1.1" 404 49  16 (io.confluent.rest-utils.requests)
schema-registry  | [2023-01-31 02:14:40,493] INFO Stopped NetworkTrafficServerConnector@2ce6c6ec{HTTP/1.1, (http/1.1)}{0.0.0.0:8081} (org.eclipse.jetty.server.AbstractConnector)
schema-registry  | [2023-01-31 02:14:40,494] INFO node0 Stopped scavenging (org.eclipse.jetty.server.session)
schema-registry  | [2023-01-31 02:14:40,496] INFO Stopped o.e.j.s.ServletContextHandler@440eaa07{/ws,null,STOPPED} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-31 02:14:40,508] INFO Stopped o.e.j.s.ServletContextHandler@7fc7c4a{/,null,STOPPED} (org.eclipse.jetty.server.handler.ContextHandler)
schema-registry  | [2023-01-31 02:14:40,509] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,510] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,510] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,510] INFO Shutting down schema registry (io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry)
schema-registry  | [2023-01-31 02:14:40,511] INFO [kafka-store-reader-thread-_schemas]: Shutting down (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,512] INFO [kafka-store-reader-thread-_schemas]: Stopped (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,516] INFO [kafka-store-reader-thread-_schemas]: Shutdown completed (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,518] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,518] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,518] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,526] INFO App info kafka.consumer for KafkaStore-reader-_schemas unregistered (org.apache.kafka.common.utils.AppInfoParser)
schema-registry  | [2023-01-31 02:14:40,526] INFO KafkaStoreReaderThread shutdown complete. (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread)
schema-registry  | [2023-01-31 02:14:40,526] INFO [Producer clientId=producer-1] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer)
schema-registry  | [2023-01-31 02:14:40,528] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,529] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,529] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
schema-registry  | [2023-01-31 02:14:40,529] INFO App info kafka.producer for producer-1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
schema-registry  | [2023-01-31 02:14:40,529] INFO Kafka store producer shut down (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-31 02:14:40,529] INFO Kafka store shut down complete (io.confluent.kafka.schemaregistry.storage.KafkaStore)
schema-registry  | [2023-01-31 02:14:40,530] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
schema-registry  | [2023-01-31 02:14:40,694] INFO Session: 0x100136f092b0007 closed (org.apache.zookeeper.ZooKeeper)
schema-registry  | [2023-01-31 02:14:40,695] INFO EventThread shut down for session: 0x100136f092b0007 (org.apache.zookeeper.ClientCnxn)
schema-registry  | ===> User
schema-registry  | uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
schema-registry  | ===> Configuring ...
schema-registry  | ===> Running preflight checks ... 
schema-registry  | ===> Check if Zookeeper is healthy ...
schema-registry  | SLF4J: Class path contains multiple SLF4J bindings.
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
schema-registry  | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
schema-registry  | log4j:WARN No appenders could be found for logger (io.confluent.admin.utils.cli.ZookeeperReadyCommand).
schema-registry  | log4j:WARN Please initialize the log4j system properly.
schema-registry  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
schema-registry  | ===> Check if Kafka is healthy ...
schema-registry  | SLF4J: Class path contains multiple SLF4J bindings.
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
schema-registry  | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
schema-registry  | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
schema-registry  | log4j:WARN No appenders could be found for logger (io.confluent.admin.utils.cli.KafkaReadyCommand).
schema-registry  | log4j:WARN Please initialize the log4j system properly.
schema-registry  | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.