datahub-project / datahub

The Metadata Platform for your Data and AI Stack
https://datahubproject.io
Apache License 2.0
9.93k stars 2.94k forks source link

Start datahub-gms appears to be stuck and failed to start #11799

Closed siatzhuliang closed 1 week ago

siatzhuliang commented 2 weeks ago

I use containers for gms, mae, mce, front, upgrade, etc., but kafka, mysql, zookeeper, etc., use existing clusters. The gms log stopped here after startup, and other services could not be successfully linked, what is the problem?

2024-11-06 17:12:41,161 [ThreadPoolTaskExecutor-1] INFO i.c.k.s.KafkaAvroDeserializerConfig:179 - KafkaAvroDeserializerConfig values: bearer.auth.token = [hidden] schema.registry.ssl.truststore.type = JKS schema.reflection = false auto.register.schemas = true basic.auth.credentials.source = URL schema.registry.ssl.keystore.password = [hidden] schema.registry.ssl.provider = schema.registry.ssl.endpoint.identification.algorithm = https schema.registry.ssl.truststore.location = specific.avro.reader = false value.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy schema.registry.url = [http://datahub-schema-registry:80] schema.registry.ssl.keystore.location = schema.registry.ssl.trustmanager.algorithm = PKIX schema.registry.ssl.key.password = [hidden] schema.registry.ssl.keystore.type = JKS proxy.port = -1 schema.registry.ssl.secure.random.implementation = schema.registry.ssl.cipher.suites = [] max.schemas.per.subject = 1000 schema.registry.ssl.truststore.password = [hidden] basic.auth.user.info = [hidden] latest.compatibility.strict = true proxy.host = use.latest.version = false schema.registry.ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] schema.registry.ssl.protocol = TLS schema.registry.basic.auth.user.info = [hidden] bearer.auth.credentials.source = STATIC_TOKEN schema.registry.ssl.keymanager.algorithm = SunX509 key.subject.name.strategy = class io.confluent.kafka.serializers.subject.TopicNameStrategy 2024-11-06 17:12:41,164 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcl-versioned' was supplied but isn't a known config. 2024-11-06 17:12:41,164 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcl-versioned' was supplied but isn't a known config. 2024-11-06 17:12:41,164 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcp.id' was supplied but isn't a known config. 2024-11-06 17:12:41,164 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcp.id' was supplied but isn't a known config. 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcl-versioned.id' was supplied but isn't a known config. 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcl-versioned.id' was supplied but isn't a known config. 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcp' was supplied but isn't a known config. 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] WARN o.a.k.c.consumer.ConsumerConfig:355 - The configuration 'data-hub.system-update.topic-key.mcp' was supplied but isn't a known config. 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] INFO o.a.kafka.common.utils.AppInfoParser:117 - Kafka version: 5.5.15-ccs 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] INFO o.a.kafka.common.utils.AppInfoParser:118 - Kafka commitId: 42c2f4267b745647 2024-11-06 17:12:41,165 [ThreadPoolTaskExecutor-1] INFO o.a.kafka.common.utils.AppInfoParser:119 - Kafka startTimeMs: 1730884361165 2024-11-06 17:12:41,170 [ThreadPoolTaskExecutor-1] INFO org.apache.kafka.clients.Metadata:277 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-8, groupId=generic-duhe-consumer-job-client-gms] Cluster ID: AnNlpbi2ShWt6QkSQP8LJQ 2024-11-06 17:12:41,170 [ThreadPoolTaskExecutor-1] INFO org.apache.kafka.clients.Metadata:277 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-8, groupId=generic-duhe-consumer-job-client-gms] Cluster ID: AnNlpbi2ShWt6QkSQP8LJQ 2024-11-06 17:12:41,299 [ThreadPoolTaskExecutor-1] INFO org.apache.kafka.clients.Metadata:277 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-8, groupId=generic-duhe-consumer-job-client-gms] Cluster ID: GoTgCgJOS1KjguCzuNHfYA 2024-11-06 17:12:41,299 [ThreadPoolTaskExecutor-1] INFO org.apache.kafka.clients.Metadata:277 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-8, groupId=generic-duhe-consumer-job-client-gms] Cluster ID: GoTgCgJOS1KjguCzuNHfYA 2024-11-06 17:12:41,411 [ThreadPoolTaskExecutor-1] INFO c.l.m.b.k.DataHubUpgradeKafkaListener:85 - Partition: DataHubUpgradeHistory_v1-2 Current Offset: 0 2024-11-06 17:12:41,412 [ThreadPoolTaskExecutor-1] INFO o.s.k.c.DefaultKafkaConsumerFactory$ExtendedKafkaConsumer:1602 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-3, groupId=generic-duhe-consumer-job-client-gms] Seeking to offset 0 for partition DataHubUpgradeHistory_v1-2 2024-11-06 17:12:41,412 [ThreadPoolTaskExecutor-1] INFO c.l.m.b.k.DataHubUpgradeKafkaListener:85 - Partition: DataHubUpgradeHistory_v1-1 Current Offset: 0 2024-11-06 17:12:41,412 [ThreadPoolTaskExecutor-1] INFO o.s.k.c.DefaultKafkaConsumerFactory$ExtendedKafkaConsumer:1602 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-3, groupId=generic-duhe-consumer-job-client-gms] Seeking to offset 0 for partition DataHubUpgradeHistory_v1-1 2024-11-06 17:12:41,413 [ThreadPoolTaskExecutor-1] INFO c.l.m.b.k.DataHubUpgradeKafkaListener:85 - Partition: DataHubUpgradeHistory_v1-0 Current Offset: 0 2024-11-06 17:12:41,413 [ThreadPoolTaskExecutor-1] INFO o.s.k.c.DefaultKafkaConsumerFactory$ExtendedKafkaConsumer:1602 - [Consumer clientId=consumer-generic-duhe-consumer-job-client-gms-3, groupId=generic-duhe-consumer-job-client-gms] Seeking to offset 0 for partition DataHubUpgradeHistory_v1-0 2024-11-06 17:12:41,415 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - generic-duhe-consumer-job-client-gms: partitions assigned: [DataHubUpgradeHistory_v1-2, DataHubUpgradeHistory_v1-1, DataHubUpgradeHistory_v1-0] 2024-11-06 17:13:17,614 [pool-11-thread-1] INFO c.d.m.ingestion.IngestionScheduler:291 - Successfully fetched 0 ingestion sources.

Looking forward to answering. Thank you

siatzhuliang commented 2 weeks ago

I've used -u RestoreIndices so far it looks OK, I'll look at it again and close this issue for now

siatzhuliang commented 2 weeks ago

It's stuck here now, and I'm not sure if the gms launched successfully

2024-11-06 19:14:22,959 [ThreadPoolTaskExecutor-1] INFO c.l.metadata.kafka.MCLKafkaListener:88 - Invoking MCL hooks for consumer: generic-mae-consumer-job-client urn: urn:li:corpuser:datahub, aspect name: corpUserEditableInfo, entity type: corpuser, change type: RESTATE 2024-11-06 19:14:22,962 [ThreadPoolTaskExecutor-1] INFO c.l.m.s.e.update.ESBulkProcessor:85 - Added request id: owfdjRxuXJ/Wjrq57Q3euA==, operation type: UPDATE, index: system_metadata_service_v1 2024-11-06 19:14:22,963 [ThreadPoolTaskExecutor-1] INFO c.l.m.s.e.update.ESBulkProcessor:85 - Added request id: urn%3Ali%3Acorpuser%3Adatahub, operation type: UPDATE, index: corpuserindex_v2 2024-11-06 19:14:22,983 [ThreadPoolTaskExecutor-1] INFO c.l.metadata.kafka.MCLKafkaListener:116 - Successfully completed MCL hooks for consumer: generic-mae-consumer-job-client urn: urn:li:corpuser:datahub 2024-11-06 19:14:23,604 [I/O dispatcher 1] INFO c.l.m.s.e.update.BulkListener:61 - Successfully fed bulk request 7. Number of events: 2 Took time ms: 637 2024-11-06 19:14:32,073 [ThreadPoolTaskExecutor-1] INFO c.l.metadata.kafka.MCLKafkaListener:88 - Invoking MCL hooks for consumer: generic-mae-consumer-job-client urn: urn:li:corpuser:datahub, aspect name: corpUserInfo, entity type: corpuser, change type: RESTATE 2024-11-06 19:14:32,075 [ThreadPoolTaskExecutor-1] INFO c.l.m.s.e.update.ESBulkProcessor:85 - Added request id: gou3MO0jzerhyT5BOUlkeg==, operation type: UPDATE, index: system_metadata_service_v1 2024-11-06 19:14:32,076 [ThreadPoolTaskExecutor-1] INFO c.l.m.s.e.update.ESBulkProcessor:85 - Added request id: urn%3Ali%3Acorpuser%3Adatahub, operation type: UPDATE, index: corpuserindex_v2 2024-11-06 19:14:32,086 [ThreadPoolTaskExecutor-1] INFO c.l.metadata.kafka.MCLKafkaListener:116 - Successfully completed MCL hooks for consumer: generic-mae-consumer-job-client urn: urn:li:corpuser:datahub 2024-11-06 19:14:32,178 [ThreadPoolTaskExecutor-1] INFO c.l.metadata.kafka.MCLKafkaListener:88 - Invoking MCL hooks for consumer: generic-mae-consumer-job-client urn: urn:li:corpuser:datahub, aspect name: corpUserKey, entity type: corpuser, change type: RESTATE 2024-11-06 19:14:32,181 [ThreadPoolTaskExecutor-1] INFO c.l.m.s.e.update.ESBulkProcessor:85 - Added request id: sRJbX/Wm3qAZ32Ql34QDLw==, operation type: UPDATE, index: system_metadata_service_v1 2024-11-06 19:14:32,181 [ThreadPoolTaskExecutor-1] INFO c.l.m.s.e.update.ESBulkProcessor:85 - Added request id: urn%3Ali%3Acorpuser%3Adatahub, operation type: UPDATE, index: corpuserindex_v2 2024-11-06 19:14:32,183 [ThreadPoolTaskExecutor-1] INFO c.l.metadata.kafka.MCLKafkaListener:116 - Successfully completed MCL hooks for consumer: generic-mae-consumer-job-client urn: urn:li:corpuser:datahub 2024-11-06 19:14:32,647 [I/O dispatcher 1] INFO c.l.m.s.e.update.BulkListener:61 - Successfully fed bulk request 8. Number of events: 2 Took time ms: 568 2024-11-06 19:14:33,672 [I/O dispatcher 1] INFO c.l.m.s.e.update.BulkListener:61 - Successfully fed bulk request 9. Number of events: 2 Took time ms: 783 2024-11-06 19:15:31,051 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:31,051 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:31,096 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Revoke previously assigned partitions MetadataChangeLog_Timeseries_v1-0, MetadataChangeLog_Versioned_v1-0 2024-11-06 19:15:31,096 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Revoke previously assigned partitions MetadataChangeLog_Timeseries_v1-0, MetadataChangeLog_Versioned_v1-0 2024-11-06 19:15:31,097 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - generic-mae-consumer-job-client: partitions revoked: [MetadataChangeLog_Timeseries_v1-0, MetadataChangeLog_Versioned_v1-0] 2024-11-06 19:15:31,097 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] (Re-)joining group 2024-11-06 19:15:31,097 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] (Re-)joining group 2024-11-06 19:15:31,124 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Finished assignment for group at generation 45: {consumer-generic-mae-consumer-job-client-1-62cb07c8-9cc5-4949-be8c-e870748cb4ee=Assignment(partitions=[MetadataChangeLog_Timeseries_v1-0, MetadataChangeLog_Versioned_v1-0]), consumer-generic-mae-consumer-job-client-1-72e47063-f791-410d-ac97-d67c137a725e=Assignment(partitions=[])} 2024-11-06 19:15:31,124 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Finished assignment for group at generation 45: {consumer-generic-mae-consumer-job-client-1-62cb07c8-9cc5-4949-be8c-e870748cb4ee=Assignment(partitions=[MetadataChangeLog_Timeseries_v1-0, MetadataChangeLog_Versioned_v1-0]), consumer-generic-mae-consumer-job-client-1-72e47063-f791-410d-ac97-d67c137a725e=Assignment(partitions=[])} 2024-11-06 19:15:31,164 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Successfully joined group with generation 45 2024-11-06 19:15:31,164 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Successfully joined group with generation 45 2024-11-06 19:15:31,165 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:31,165 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-generic-mae-consumer-job-client-1, groupId=generic-mae-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:31,165 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - generic-mae-consumer-job-client: partitions assigned: [] 2024-11-06 19:15:31,906 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:31,906 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:31,952 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Revoke previously assigned partitions DataHubUsageEvent_v1-0 2024-11-06 19:15:31,952 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Revoke previously assigned partitions DataHubUsageEvent_v1-0 2024-11-06 19:15:31,953 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - datahub-usage-event-consumer-job-client: partitions revoked: [DataHubUsageEvent_v1-0] 2024-11-06 19:15:31,953 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] (Re-)joining group 2024-11-06 19:15:31,953 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] (Re-)joining group 2024-11-06 19:15:31,983 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Finished assignment for group at generation 44: {consumer-datahub-usage-event-consumer-job-client-5-7a753ab8-865e-4e47-a024-3e627949edf7=Assignment(partitions=[]), consumer-datahub-usage-event-consumer-job-client-2-e12c9d81-4108-414f-a0ff-9058f345a1dc=Assignment(partitions=[DataHubUsageEvent_v1-0])} 2024-11-06 19:15:31,983 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Finished assignment for group at generation 44: {consumer-datahub-usage-event-consumer-job-client-5-7a753ab8-865e-4e47-a024-3e627949edf7=Assignment(partitions=[]), consumer-datahub-usage-event-consumer-job-client-2-e12c9d81-4108-414f-a0ff-9058f345a1dc=Assignment(partitions=[DataHubUsageEvent_v1-0])} 2024-11-06 19:15:32,029 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Successfully joined group with generation 44 2024-11-06 19:15:32,029 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Successfully joined group with generation 44 2024-11-06 19:15:32,030 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:32,030 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-datahub-usage-event-consumer-job-client-5, groupId=datahub-usage-event-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:32,030 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - datahub-usage-event-consumer-job-client: partitions assigned: [] 2024-11-06 19:15:33,985 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:33,985 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:34,029 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Revoke previously assigned partitions MetadataChangeProposal_v1-0 2024-11-06 19:15:34,029 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Revoke previously assigned partitions MetadataChangeProposal_v1-0 2024-11-06 19:15:34,029 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - generic-mce-consumer-job-client: partitions revoked: [MetadataChangeProposal_v1-0] 2024-11-06 19:15:34,029 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] (Re-)joining group 2024-11-06 19:15:34,029 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] (Re-)joining group 2024-11-06 19:15:34,058 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Finished assignment for group at generation 47: {consumer-generic-mce-consumer-job-client-4-fc9056d3-cdc9-4770-ad9d-f34f753c8fde=Assignment(partitions=[]), consumer-generic-mce-consumer-job-client-2-c992714a-4a30-4dab-9130-adb82f92264e=Assignment(partitions=[MetadataChangeProposal_v1-0])} 2024-11-06 19:15:34,058 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Finished assignment for group at generation 47: {consumer-generic-mce-consumer-job-client-4-fc9056d3-cdc9-4770-ad9d-f34f753c8fde=Assignment(partitions=[]), consumer-generic-mce-consumer-job-client-2-c992714a-4a30-4dab-9130-adb82f92264e=Assignment(partitions=[MetadataChangeProposal_v1-0])} 2024-11-06 19:15:34,102 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Successfully joined group with generation 47 2024-11-06 19:15:34,102 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Successfully joined group with generation 47 2024-11-06 19:15:34,102 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:34,102 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-generic-mce-consumer-job-client-4, groupId=generic-mce-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:34,102 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - generic-mce-consumer-job-client: partitions assigned: [] 2024-11-06 19:15:34,115 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:34,115 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:1054 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Attempt to heartbeat failed since group is rebalancing 2024-11-06 19:15:34,154 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Revoke previously assigned partitions MetadataChangeEvent_v4-0 2024-11-06 19:15:34,154 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:294 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Revoke previously assigned partitions MetadataChangeEvent_v4-0 2024-11-06 19:15:34,154 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - mce-consumer-job-client: partitions revoked: [MetadataChangeEvent_v4-0] 2024-11-06 19:15:34,154 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] (Re-)joining group 2024-11-06 19:15:34,154 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:552 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] (Re-)joining group 2024-11-06 19:15:34,179 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Finished assignment for group at generation 47: {consumer-mce-consumer-job-client-1-ea9d282b-2fde-4349-beb2-74743c05f2a7=Assignment(partitions=[MetadataChangeEvent_v4-0]), consumer-mce-consumer-job-client-2-98c1e4f6-a281-4ab4-81ef-6ec51f85f72d=Assignment(partitions=[])} 2024-11-06 19:15:34,179 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:616 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Finished assignment for group at generation 47: {consumer-mce-consumer-job-client-1-ea9d282b-2fde-4349-beb2-74743c05f2a7=Assignment(partitions=[MetadataChangeEvent_v4-0]), consumer-mce-consumer-job-client-2-98c1e4f6-a281-4ab4-81ef-6ec51f85f72d=Assignment(partitions=[])} 2024-11-06 19:15:34,218 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Successfully joined group with generation 47 2024-11-06 19:15:34,218 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.AbstractCoordinator:503 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Successfully joined group with generation 47 2024-11-06 19:15:34,219 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:34,219 [ThreadPoolTaskExecutor-1] INFO o.a.k.c.c.i.ConsumerCoordinator:275 - [Consumer clientId=consumer-mce-consumer-job-client-2, groupId=mce-consumer-job-client] Adding newly assigned partitions: 2024-11-06 19:15:34,219 [ThreadPoolTaskExecutor-1] INFO o.s.k.l.KafkaMessageListenerContainer:292 - mce-consumer-job-client: partitions assigned: []

siatzhuliang commented 2 weeks ago

Finally, I successfully started by turning off the dependency update service