MySQL CDC作为source&emqx作为sink一直报错【emqx模块】

20100507 commented 1 year ago

Search before asking

[ ] I had searched in the issues and found no similar question.
[ ] I had googled my question but i didn't get any help.
[ ] I had read the documentation: ChunJun doc but it didn't help me.

Description

版本 master分支环境 flink1.16.1、flink cdc2.3 代码 `set execution.checkpointing.interval=1s;

CREATE TABLE source( value FLOAT , count int, name STRING, PRIMARY KEY (count) NOT ENFORCED ) WITH ( 'connector' = 'mysql-cdc', 'hostname' = 'xxxxx', 'port' = '19901', 'database-name' = 'test', 'table-name' = 'test', 'username' = 'root', 'password' = '123456789');

CREATE TABLE sink_ods ( value FLOAT, count int, name STRING ) WITH ( 'connector' = 'emqx-x' ,'broker' = 'tcp://xxx:1883' ,'topic' = 'cx' ,'isCleanSession' = 'true' ,'qos' = '2' ,'username' = 'admin' ,'password' = 'public' ,'format' = 'json' ); insert into sink_ods select * from source;`

上游表结构

CREATE TABLEtest( valuefloat NOT NULL, countint(11) NOT NULL, namevarchar(255) CHARACTER SET utf8 DEFAULT NULL, PRIMARY KEY (count) ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;

报错内容如下

org.apache.flink.runtime.JobException: The failure is not recoverable at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:120) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:83) at org.apache.flink.runtime.scheduler.DefaultScheduler.recordTaskFailure(DefaultScheduler.java:256) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:247) at org.apache.flink.runtime.scheduler.DefaultScheduler.onTaskFailed(DefaultScheduler.java:240) at org.apache.flink.runtime.scheduler.SchedulerBase.onTaskExecutionStateUpdate(SchedulerBase.java:738) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:715) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:78) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:477) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRpcInvocation$1(AkkaRpcActor.java:309) at org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:83) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:307) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:222) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:84) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:168) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580) at akka.actor.ActorCell.invoke(ActorCell.scala:548) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) Caused by: com.dtstack.chunjun.throwable.NoRestartException: The dirty consumer shutdown, due to the consumed count exceed the max-consumed [0] at com.dtstack.chunjun.dirty.consumer.DirtyDataCollector.addConsumed(DirtyDataCollector.java:99) at com.dtstack.chunjun.dirty.consumer.DirtyDataCollector.offer(DirtyDataCollector.java:73) at com.dtstack.chunjun.dirty.manager.DirtyManager.collect(DirtyManager.java:134) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:490) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:302) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:89) at com.dtstack.chunjun.sink.DtOutputFormatSinkFunction.invoke(DtOutputFormatSinkFunction.java:112) at org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:65) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56) at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29) at org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:39) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:313) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter$OutputCollector.collect(MySqlRecordEmitter.java:154) at com.ververica.cdc.debezium.table.RowDataDebeziumDeserializeSchema.emit(RowDataDebeziumDeserializeSchema.java:162) at com.ververica.cdc.debezium.table.RowDataDebeziumDeserializeSchema.deserialize(RowDataDebeziumDeserializeSchema.java:127) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitElement(MySqlRecordEmitter.java:129) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.processElement(MySqlRecordEmitter.java:111) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:83) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55) at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:143) at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385) at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) at java.lang.Thread.run(Thread.java:748)

Code of Conduct

[X] I agree to follow this project's Code of Conduct

20100507 commented 1 year ago

补充异常信息 2023-05-17 11:24:12,573 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -------------------------------------------------------------------------------- 2023-05-17 11:24:12,576 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Starting YARN TaskExecutor runner (Version: 1.16.1, Scala: 2.12, Rev:DeadD0d0, Date:1970-01-01T01:00:00+01:00) 2023-05-17 11:24:12,577 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - OS current user: root 2023-05-17 11:24:12,724 WARN org.apache.hadoop.util.NativeCodeLoader [] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-05-17 11:24:12,783 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Current Hadoop/Kerberos user: root 2023-05-17 11:24:12,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - JVM: Java HotSpot(TM) 64-Bit Server VM - Oracle Corporation - 1.8/25.211-b12 2023-05-17 11:24:12,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Arch: amd64 2023-05-17 11:24:12,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Maximum heap size: 4002 MiBytes 2023-05-17 11:24:12,784 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - JAVA_HOME: /soft/jdk1.8.0_211 2023-05-17 11:24:12,785 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Hadoop version: 3.2.2 2023-05-17 11:24:12,785 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - JVM Options: 2023-05-17 11:24:12,785 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Xmx4378853376 2023-05-17 11:24:12,785 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Xms4378853376 2023-05-17 11:24:12,785 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -XX:MaxDirectMemorySize=2206203904 2023-05-17 11:24:12,785 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -XX:MaxMetaspaceSize=134217728 2023-05-17 11:24:12,786 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dlog.file=/soft/hadoop/logs/userlogs/application_1681371658709_0339/container_1681371658709_0339_01_000002/taskmanager.log 2023-05-17 11:24:12,786 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dlog4j.configuration=file:./log4j.properties 2023-05-17 11:24:12,786 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dlog4j.configurationFile=file:./log4j.properties 2023-05-17 11:24:12,786 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Program Arguments: 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.network.min=2147483648b 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.cpu.cores=4.0 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.task.off-heap.size=29360128b 2023-05-17 11:24:12,787 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.jvm-metaspace.size=134217728b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - external-resources=none 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.jvm-overhead.min=754042664b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.framework.off-heap.size=29360128b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.network.max=2147483648b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.framework.heap.size=83886080b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.managed.size=67108864b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.task.heap.size=4294967296b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.numberOfTaskSlots=4 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -D 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - taskmanager.memory.jvm-overhead.max=754042664b 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - --configDir 2023-05-17 11:24:12,788 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - . 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.rpc.address=jykj2.yarn.com 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dweb.port=0 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.memory.off-heap.size=67108864b 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dweb.tmpdir=/tmp/flink-web-00bebafd-e653-4804-b8a0-3daf02dfdee5 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.rpc.port=43740 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Drest.bind-address=jykj2.yarn.com 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Drest.address=jykj2.yarn.com 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.memory.jvm-overhead.max=82021946b 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dblob.server.port=40537 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.memory.jvm-overhead.min=82021946b 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dtaskmanager.resource-id=container_1681371658709_0339_01_000002 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Dinternal.taskmanager.resource-id.metadata=jykj2.yarn.com:36423 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.memory.jvm-metaspace.size=134217728b 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -Djobmanager.memory.heap.size=536870912b 2023-05-17 11:24:12,789 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Classpath: :lib/chunjun-connector-emqx-master.jar:lib/chunjun-connector-jdbc-base-master.jar:lib/chunjun-connector-kingbase-master.jar:lib/chunjun-connector-sqlserver-master.jar:lib/chunjun-core-master.jar:lib/chunjun-dirty-log-master.jar:lib/flink-cep-1.16.1.jar:lib/flink-connector-api-1.5.0.jar:lib/flink-connector-files-1.16.1.jar:lib/flink-connector-jdbc-1.16.0.jar:lib/flink-connector-jdbc-3.1-SNAPSHOT.jar:lib/flink-csv-1.16.1.jar:lib/flink-json-1.16.1.jar:lib/flink-scala_2.12-1.16.1.jar:lib/flink-shaded-zookeeper-3.5.9.jar:lib/flink-sql-connector-kafka-1.16.1.jar:lib/flink-sql-connector-mongodb-1.0.0-1.16.jar:lib/flink-sql-connector-mongodb-cdc-2.3.0.jar:lib/flink-sql-connector-mysql-cdc-2.3.0.jar:lib/flink-sql-connector-oracle-cdc-2.3.0.jar:lib/flink-sql-connector-postgres-cdc-2.3.0.jar:lib/flink-sql-connector-sqlserver-cdc-2.3.0.jar:lib/flink-table-api-java-uber-1.16.1.jar:lib/flink-table-planner-loader-1.16.1.jar:lib/flink-table-runtime-1.16.1.jar:lib/hadoop-mapreduce-client-core-3.2.2.jar:lib/iceberg-flink-1.13-runtime-0.12.0-pr3095-20220823.jar:lib/jtds-1.3.1.jar:lib/log4j-1.2-api-2.17.1.jar:lib/log4j-api-2.17.1.jar:lib/log4j-core-2.17.1.jar:lib/log4j-slf4j-impl-2.17.1.jar:lib/mysql-connector-java-5.1.35.jar:lib/org.eclipse.paho.client.mqttv3-1.2.2.jar:flink-dist-1.16.1.jar:flink-conf.yaml::/soft/hadoop/etc/hadoop:/soft/hadoop/share/hadoop/common/hadoop-common-3.2.2-tests.jar:/soft/hadoop/share/hadoop/common/hadoop-common-3.2.2.jar:/soft/hadoop/share/hadoop/common/hadoop-kms-3.2.2.jar:/soft/hadoop/share/hadoop/common/hadoop-nfs-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/accessors-smart-1.2.jar:/soft/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/soft/hadoop/share/hadoop/common/lib/asm-5.0.4.jar:/soft/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/soft/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/soft/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/soft/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/soft/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/soft/hadoop/share/hadoop/common/lib/commons-codec-1.11.jar:/soft/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/commons-compress-1.19.jar:/soft/hadoop/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/soft/hadoop/share/hadoop/common/lib/commons-io-2.5.jar:/soft/hadoop/share/hadoop/common/lib/commons-lang3-3.7.jar:/soft/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/soft/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/soft/hadoop/share/hadoop/common/lib/commons-net-3.6.jar:/soft/hadoop/share/hadoop/common/lib/commons-text-1.4.jar:/soft/hadoop/share/hadoop/common/lib/curator-client-2.13.0.jar:/soft/hadoop/share/hadoop/common/lib/curator-framework-2.13.0.jar:/soft/hadoop/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/soft/hadoop/share/hadoop/common/lib/dnsjava-2.1.7.jar:/soft/hadoop/share/hadoop/common/lib/error_prone_annotations-2.2.0.jar:/soft/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/soft/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/soft/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/soft/hadoop/share/hadoop/common/lib/hadoop-annotations-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/hadoop-auth-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/soft/hadoop/share/hadoop/common/lib/httpclient-4.5.13.jar:/soft/hadoop/share/hadoop/common/lib/httpcore-4.4.13.jar:/soft/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/soft/hadoop/share/hadoop/common/lib/jackson-annotations-2.9.10.jar:/soft/hadoop/share/hadoop/common/lib/jackson-core-2.9.10.jar:/soft/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/jackson-databind-2.9.10.4.jar:/soft/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/javax.activation-api-1.2.0.jar:/soft/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/soft/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/soft/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/soft/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/soft/hadoop/share/hadoop/common/lib/jersey-core-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jersey-json-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jersey-server-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/soft/hadoop/share/hadoop/common/lib/jetty-http-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-io-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-security-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-server-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-servlet-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-util-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-webapp-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-xml-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jsch-0.1.55.jar:/soft/hadoop/share/hadoop/common/lib/json-smart-2.3.jar:/soft/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/soft/hadoop/share/hadoop/common/lib/jsr305-3.0.2.jar:/soft/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/soft/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/soft/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/soft/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/soft/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/soft/hadoop/share/hadoop/common/lib/netty-3.10.6.Final.jar:/soft/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-7.9.jar:/soft/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/soft/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/soft/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/soft/hadoop/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/soft/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/soft/hadoop/share/hadoop/common/lib/snappy-java-1.0.5.jar:/soft/hadoop/share/hadoop/common/lib/stax2-api-3.1.4.jar:/soft/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/soft/hadoop/share/hadoop/common/lib/zookeeper-3.4.13.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/soft/hadoop/share/hadoop/hdfs/lib/asm-5.0.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/soft/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-compress-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-io-2.5.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-net-3.6.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-text-1.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/soft/hadoop/share/hadoop/hdfs/lib/error_prone_annotations-2.2.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/gson-2.2.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/soft/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/soft/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.9.10.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-core-2.9.10.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.9.10.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/javax.activation-api-1.2.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/soft/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jettison-1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-http-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-io-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-security-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-server-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-util-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/soft/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/json-smart-2.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/soft/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/soft/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/soft/hadoop/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/soft/hadoop/share/hadoop/hdfs/lib/netty-all-4.1.48.Final.jar:/soft/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-7.9.jar:/soft/hadoop/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/soft/hadoop/share/hadoop/hdfs/lib/okio-1.6.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/soft/hadoop/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-submarine-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/soft/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/soft/hadoop/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/soft/hadoop/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/soft/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/soft/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/soft/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/soft/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/soft/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/soft/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.9.10.jar:/soft/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.9.10.jar:/soft/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.9.10.jar:/soft/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/soft/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/soft/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.jar:/soft/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/soft/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/soft/hadoop/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/soft/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/soft/hadoop/share/hadoop/yarn/lib/objenesis-1.0.jar:/soft/hadoop/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/soft/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar 2023-05-17 11:24:12,790 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - -------------------------------------------------------------------------------- 2023-05-17 11:24:12,791 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Registered UNIX signal handlers for [TERM, HUP, INT] 2023-05-17 11:24:12,795 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Current working Directory: /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/container_1681371658709_0339_01_000002 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: historyserver.web.tmpdir, /tmp/flinkhistoryserver/ 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: yarn.flink-dist-jar, file:/soft/flink-1.16.1/lib/flink-dist-1.16.1.jar 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.cluster-id, application_1681371658709_0339 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.rpc.address, localhost 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.task.off-heap.size, 28m 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.jvm-metaspace.size, 128m 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.storageDir, hdfs:///flink-zk/ha/ 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: task.cancellation.timeout, 180000 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: parallelism.default, 1 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: historyserver.archive.fs.dir, hdfs://jykj0.yarn.com:9000/completed-jobs/ 2023-05-17 11:24:12,812 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.off-heap.size, 64m 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.backend.incremental, true 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.network.fraction, 0.5 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: historyserver.archive.fs.refresh-interval, 10000 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.rpc.port, 6123 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: yarn.properties-file.location, /tmp/flink-1.16.1 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.zookeeper.quorum, jykj0.yarn.com:2181,jykj1.yarn.com:2181,jykj2.yarn.com:2181 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: yarn.application-attempt-failures-validity-interval, 1200000 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: restart-strategy.fixed-delay.delay, 10 s 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.backend, rocksdb 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability.zookeeper.client.acl, open 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: $internal.deployment.config-dir, /soft/flink-1.16.1/conf 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: $internal.yarn.log-config-file, /soft/flink-1.16.1/conf/log4j.properties 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.jvm-overhead.max, 128m 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: containerized.heap-cutoff-min, 100mb 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: historyserver.web.address, 172.24.1.135 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: historyserver.web.port, 8082 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: classloader.check-leaked-classloader, false 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: pipeline.time-characteristic, EventTime 2023-05-17 11:24:12,813 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.execution.failover-strategy, region 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.jvm-overhead.min, 64m 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.savepoints.dir, hdfs://jykj0.yarn.com:9000/flink-savepoints 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.jvm-overhead.min, 64m 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.framework.off-heap.size, 28m 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: yarn.application-attempts, 3 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.task.heap.size, 4096m 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.numberOfTaskSlots, 4 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.jvm-overhead.max, 1280m 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.archive.fs.dir, hdfs://jykj0.yarn.com:9000/completed-jobs/ 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: heartbeat.timeout, 180000 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.network.min, 64mb 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: classloader.resolve-order, parent-first 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: restart-strategy.fixed-delay.attempts, 3 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: execution.target, yarn-session 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.network.max, 2048mb 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: containerized.heap-cutoff-ratio, 0.1 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: execution.attached, false 2023-05-17 11:24:12,814 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: internal.cluster.execution-mode, NORMAL 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.framework.heap.size, 80m 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: taskmanager.memory.managed.size, 64m 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: high-availability, zookeeper 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: restart-strategy, fixed-delay 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.jvm-metaspace.size, 128m 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: jobmanager.memory.heap.size, 512m 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading configuration property: state.checkpoints.dir, hdfs://jykj0.yarn.com:9000/flink-checkpoints 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: blob.server.port, 40537 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.rpc.address, jykj2.yarn.com 2023-05-17 11:24:12,815 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.jvm-metaspace.size, 134217728b 2023-05-17 11:24:12,816 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.cpu.cores, 4.0 2023-05-17 11:24:12,816 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.task.off-heap.size, 29360128b 2023-05-17 11:24:12,816 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.memory.jvm-overhead.min, 82021946b 2023-05-17 11:24:12,816 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: external-resources, none 2023-05-17 11:24:12,816 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.jvm-overhead.min, 754042664b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.framework.off-heap.size, 29360128b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.numberOfTaskSlots, 4 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.task.heap.size, 4294967296b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.jvm-overhead.max, 754042664b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.resource-id, container_1681371658709_0339_01_000002 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: web.port, 0 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.network.min, 2147483648b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.memory.off-heap.size, 67108864b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: web.tmpdir, /tmp/flink-web-00bebafd-e653-4804-b8a0-3daf02dfdee5 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: internal.taskmanager.resource-id.metadata, jykj2.yarn.com:36423 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.rpc.port, 43740 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: rest.bind-address, jykj2.yarn.com 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.network.max, 2147483648b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.managed.size, 67108864b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: taskmanager.memory.framework.heap.size, 83886080b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: rest.address, jykj2.yarn.com 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.memory.jvm-metaspace.size, 134217728b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.memory.heap.size, 536870912b 2023-05-17 11:24:12,817 INFO org.apache.flink.configuration.GlobalConfiguration [] - Loading dynamic configuration property: jobmanager.memory.jvm-overhead.max, 82021946b 2023-05-17 11:24:12,818 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - Current working/local Directory: /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339 2023-05-17 11:24:12,832 INFO org.apache.flink.runtime.clusterframework.BootstrapTools [] - Setting directories for temporary files to: /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339 2023-05-17 11:24:12,832 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - TM: local keytab path obtained null 2023-05-17 11:24:12,832 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - TM: keytab principal obtained null 2023-05-17 11:24:12,847 INFO org.apache.flink.yarn.YarnTaskExecutorRunner [] - YARN daemon is running as: root Yarn client user obtainer: root 2023-05-17 11:24:12,924 INFO org.apache.flink.runtime.state.changelog.StateChangelogStorageLoader [] - StateChangelogStorageLoader initialized with shortcut names {memory,filesystem}. 2023-05-17 11:24:12,926 INFO org.apache.flink.runtime.state.changelog.StateChangelogStorageLoader [] - StateChangelogStorageLoader initialized with shortcut names {memory,filesystem}. 2023-05-17 11:24:12,957 INFO org.apache.flink.runtime.security.modules.HadoopModule [] - Hadoop user set to root (auth:SIMPLE) 2023-05-17 11:24:12,970 INFO org.apache.flink.runtime.security.modules.JaasModule [] - Jaas file will be created as /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/jaas-3679752976072548338.conf. 2023-05-17 11:24:13,740 INFO org.apache.flink.runtime.blob.FileSystemBlobStore [] - Creating highly available BLOB storage directory at hdfs:/flink-zk/ha/application_1681371658709_0339/blob 2023-05-17 11:24:13,803 INFO org.apache.flink.runtime.util.ZooKeeperUtils [] - Enforcing default ACL for ZK connections 2023-05-17 11:24:13,805 INFO org.apache.flink.runtime.util.ZooKeeperUtils [] - Using '/flink/application_1681371658709_0339' as Zookeeper namespace. 2023-05-17 11:24:13,870 INFO org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl [] - Starting 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:zookeeper.version=3.5.9-83df9301aa5c2a5d284a9940177808c01bc35cef, built on 01/06/2021 20:03 GMT 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:host.name=jykj2.yarn.com 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.version=1.8.0_211 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.vendor=Oracle Corporation 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.home=/soft/jdk1.8.0_211/jre 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.class.path=:lib/chunjun-connector-emqx-master.jar:lib/chunjun-connector-jdbc-base-master.jar:lib/chunjun-connector-kingbase-master.jar:lib/chunjun-connector-sqlserver-master.jar:lib/chunjun-core-master.jar:lib/chunjun-dirty-log-master.jar:lib/flink-cep-1.16.1.jar:lib/flink-connector-api-1.5.0.jar:lib/flink-connector-files-1.16.1.jar:lib/flink-connector-jdbc-1.16.0.jar:lib/flink-connector-jdbc-3.1-SNAPSHOT.jar:lib/flink-csv-1.16.1.jar:lib/flink-json-1.16.1.jar:lib/flink-scala_2.12-1.16.1.jar:lib/flink-shaded-zookeeper-3.5.9.jar:lib/flink-sql-connector-kafka-1.16.1.jar:lib/flink-sql-connector-mongodb-1.0.0-1.16.jar:lib/flink-sql-connector-mongodb-cdc-2.3.0.jar:lib/flink-sql-connector-mysql-cdc-2.3.0.jar:lib/flink-sql-connector-oracle-cdc-2.3.0.jar:lib/flink-sql-connector-postgres-cdc-2.3.0.jar:lib/flink-sql-connector-sqlserver-cdc-2.3.0.jar:lib/flink-table-api-java-uber-1.16.1.jar:lib/flink-table-planner-loader-1.16.1.jar:lib/flink-table-runtime-1.16.1.jar:lib/hadoop-mapreduce-client-core-3.2.2.jar:lib/iceberg-flink-1.13-runtime-0.12.0-pr3095-20220823.jar:lib/jtds-1.3.1.jar:lib/log4j-1.2-api-2.17.1.jar:lib/log4j-api-2.17.1.jar:lib/log4j-core-2.17.1.jar:lib/log4j-slf4j-impl-2.17.1.jar:lib/mysql-connector-java-5.1.35.jar:lib/org.eclipse.paho.client.mqttv3-1.2.2.jar:flink-dist-1.16.1.jar:flink-conf.yaml::/soft/hadoop/etc/hadoop:/soft/hadoop/share/hadoop/common/hadoop-common-3.2.2-tests.jar:/soft/hadoop/share/hadoop/common/hadoop-common-3.2.2.jar:/soft/hadoop/share/hadoop/common/hadoop-kms-3.2.2.jar:/soft/hadoop/share/hadoop/common/hadoop-nfs-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/accessors-smart-1.2.jar:/soft/hadoop/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/soft/hadoop/share/hadoop/common/lib/asm-5.0.4.jar:/soft/hadoop/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/soft/hadoop/share/hadoop/common/lib/avro-1.7.7.jar:/soft/hadoop/share/hadoop/common/lib/checker-qual-2.5.2.jar:/soft/hadoop/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/soft/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/soft/hadoop/share/hadoop/common/lib/commons-codec-1.11.jar:/soft/hadoop/share/hadoop/common/lib/commons-collections-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/commons-compress-1.19.jar:/soft/hadoop/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/soft/hadoop/share/hadoop/common/lib/commons-io-2.5.jar:/soft/hadoop/share/hadoop/common/lib/commons-lang3-3.7.jar:/soft/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/soft/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/soft/hadoop/share/hadoop/common/lib/commons-net-3.6.jar:/soft/hadoop/share/hadoop/common/lib/commons-text-1.4.jar:/soft/hadoop/share/hadoop/common/lib/curator-client-2.13.0.jar:/soft/hadoop/share/hadoop/common/lib/curator-framework-2.13.0.jar:/soft/hadoop/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/soft/hadoop/share/hadoop/common/lib/dnsjava-2.1.7.jar:/soft/hadoop/share/hadoop/common/lib/error_prone_annotations-2.2.0.jar:/soft/hadoop/share/hadoop/common/lib/failureaccess-1.0.jar:/soft/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/soft/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar:/soft/hadoop/share/hadoop/common/lib/hadoop-annotations-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/hadoop-auth-3.2.2.jar:/soft/hadoop/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/soft/hadoop/share/hadoop/common/lib/httpclient-4.5.13.jar:/soft/hadoop/share/hadoop/common/lib/httpcore-4.4.13.jar:/soft/hadoop/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/soft/hadoop/share/hadoop/common/lib/jackson-annotations-2.9.10.jar:/soft/hadoop/share/hadoop/common/lib/jackson-core-2.9.10.jar:/soft/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/jackson-databind-2.9.10.4.jar:/soft/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/soft/hadoop/share/hadoop/common/lib/javax.activation-api-1.2.0.jar:/soft/hadoop/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/soft/hadoop/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/soft/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/soft/hadoop/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/soft/hadoop/share/hadoop/common/lib/jersey-core-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jersey-json-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jersey-server-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jersey-servlet-1.19.jar:/soft/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/soft/hadoop/share/hadoop/common/lib/jetty-http-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-io-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-security-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-server-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-servlet-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-util-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-webapp-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jetty-xml-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/common/lib/jsch-0.1.55.jar:/soft/hadoop/share/hadoop/common/lib/json-smart-2.3.jar:/soft/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/soft/hadoop/share/hadoop/common/lib/jsr305-3.0.2.jar:/soft/hadoop/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/soft/hadoop/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/soft/hadoop/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-client-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-common-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-core-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-server-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerb-util-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-config-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-util-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/soft/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/soft/hadoop/share/hadoop/common/lib/metrics-core-3.2.4.jar:/soft/hadoop/share/hadoop/common/lib/netty-3.10.6.Final.jar:/soft/hadoop/share/hadoop/common/lib/nimbus-jose-jwt-7.9.jar:/soft/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/soft/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/soft/hadoop/share/hadoop/common/lib/re2j-1.1.jar:/soft/hadoop/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/soft/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/soft/hadoop/share/hadoop/common/lib/snappy-java-1.0.5.jar:/soft/hadoop/share/hadoop/common/lib/stax2-api-3.1.4.jar:/soft/hadoop/share/hadoop/common/lib/token-provider-1.0.1.jar:/soft/hadoop/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/soft/hadoop/share/hadoop/common/lib/zookeeper-3.4.13.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-client-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-native-client-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.2-tests.jar:/soft/hadoop/share/hadoop/hdfs/hadoop-hdfs-rbf-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/soft/hadoop/share/hadoop/hdfs/lib/asm-5.0.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/avro-1.7.7.jar:/soft/hadoop/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-compress-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-io-2.5.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-lang3-3.7.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-net-3.6.jar:/soft/hadoop/share/hadoop/hdfs/lib/commons-text-1.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/dnsjava-2.1.7.jar:/soft/hadoop/share/hadoop/hdfs/lib/error_prone_annotations-2.2.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/gson-2.2.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/soft/hadoop/share/hadoop/hdfs/lib/hadoop-annotations-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/hadoop-auth-3.2.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/soft/hadoop/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-annotations-2.9.10.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-core-2.9.10.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-databind-2.9.10.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/soft/hadoop/share/hadoop/hdfs/lib/javax.activation-api-1.2.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/soft/hadoop/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/soft/hadoop/share/hadoop/hdfs/lib/jettison-1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-http-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-io-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-security-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-server-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-servlet-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-util-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-webapp-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jetty-xml-9.4.20.v20190813.jar:/soft/hadoop/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/soft/hadoop/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/json-smart-2.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/soft/hadoop/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/soft/hadoop/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/soft/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/soft/hadoop/share/hadoop/hdfs/lib/netty-3.10.6.Final.jar:/soft/hadoop/share/hadoop/hdfs/lib/netty-all-4.1.48.Final.jar:/soft/hadoop/share/hadoop/hdfs/lib/nimbus-jose-jwt-7.9.jar:/soft/hadoop/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/soft/hadoop/share/hadoop/hdfs/lib/okio-1.6.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/paranamer-2.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/soft/hadoop/share/hadoop/hdfs/lib/re2j-1.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/soft/hadoop/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/soft/hadoop/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/soft/hadoop/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/soft/hadoop/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-api-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-client-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-common-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-registry-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-common-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-router-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-tests-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-services-api-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-services-core-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/hadoop-yarn-submarine-3.2.2.jar:/soft/hadoop/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/soft/hadoop/share/hadoop/yarn/lib/aopalliance-1.0.jar:/soft/hadoop/share/hadoop/yarn/lib/bcpkix-jdk15on-1.60.jar:/soft/hadoop/share/hadoop/yarn/lib/bcprov-jdk15on-1.60.jar:/soft/hadoop/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/soft/hadoop/share/hadoop/yarn/lib/fst-2.50.jar:/soft/hadoop/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/soft/hadoop/share/hadoop/yarn/lib/guice-4.0.jar:/soft/hadoop/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/soft/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-base-2.9.10.jar:/soft/hadoop/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.9.10.jar:/soft/hadoop/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.9.10.jar:/soft/hadoop/share/hadoop/yarn/lib/java-util-1.9.0.jar:/soft/hadoop/share/hadoop/yarn/lib/javax.inject-1.jar:/soft/hadoop/share/hadoop/yarn/lib/jersey-client-1.19.jar:/soft/hadoop/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/soft/hadoop/share/hadoop/yarn/lib/json-io-2.5.1.jar:/soft/hadoop/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/soft/hadoop/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/soft/hadoop/share/hadoop/yarn/lib/objenesis-1.0.jar:/soft/hadoop/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/soft/hadoop/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.io.tmpdir=/tmp 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:java.compiler=<NA> 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:os.name=Linux 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:os.arch=amd64 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:os.version=3.10.0-957.el7.x86_64 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:user.name=root 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:user.home=/root 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:user.dir=/hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/container_1681371658709_0339_01_000002 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:os.memory.free=3791MB 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:os.memory.max=4002MB 2023-05-17 11:24:13,876 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Client environment:os.memory.total=4002MB 2023-05-17 11:24:13,878 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ZooKeeper [] - Initiating client connection, connectString=jykj0.yarn.com:2181,jykj1.yarn.com:2181,jykj2.yarn.com:2181 sessionTimeout=60000 watcher=org.apache.flink.shaded.curator5.org.apache.curator.ConnectionState@5d10455d 2023-05-17 11:24:13,882 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.common.X509Util [] - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation 2023-05-17 11:24:13,885 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxnSocket [] - jute.maxbuffer value is 4194304 Bytes 2023-05-17 11:24:13,892 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - zookeeper.request.timeout value is 0. feature enabled= 2023-05-17 11:24:13,898 WARN org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/jaas-3679752976072548338.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. 2023-05-17 11:24:13,899 INFO org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.CuratorFrameworkImpl [] - Default schema 2023-05-17 11:24:13,900 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Opening socket connection to server jykj0.yarn.com/172.24.1.135:2181 2023-05-17 11:24:13,900 ERROR org.apache.flink.shaded.curator5.org.apache.curator.ConnectionState [] - Authentication failed 2023-05-17 11:24:13,901 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Socket connection established, initiating session, client: /172.24.1.137:41832, server: jykj0.yarn.com/172.24.1.135:2181 2023-05-17 11:24:13,906 INFO org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.ClientCnxn [] - Session establishment complete on server jykj0.yarn.com/172.24.1.135:2181, sessionid = 0x1019ff415800988, negotiated timeout = 40000 2023-05-17 11:24:13,907 INFO org.apache.flink.shaded.curator5.org.apache.curator.framework.state.ConnectionStateManager [] - State change: CONNECTED 2023-05-17 11:24:13,921 INFO org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.EnsembleTracker [] - New config event received: {server.1=172.24.1.135:2888:3888:participant, version=0, server.3=172.24.1.137:2888:3888:participant, server.2=172.24.1.136:2888:3888:participant} 2023-05-17 11:24:13,925 INFO org.apache.flink.shaded.curator5.org.apache.curator.framework.imps.EnsembleTracker [] - New config event received: {server.1=172.24.1.135:2888:3888:participant, version=0, server.3=172.24.1.137:2888:3888:participant, server.2=172.24.1.136:2888:3888:participant} 2023-05-17 11:24:14,101 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Using configured hostname/address for TaskManager: jykj2.yarn.com. 2023-05-17 11:24:14,176 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to start actor system, external address jykj2.yarn.com:0, bind address 0.0.0.0:0. 2023-05-17 11:24:14,969 INFO akka.event.slf4j.Slf4jLogger [] - Slf4jLogger started 2023-05-17 11:24:14,994 INFO akka.remote.RemoteActorRefProvider [] - Akka Cluster not in use - enabling unsafe features anyway becauseakka.remote.use-unsafe-remote-features-outside-clusterhas been enabled. 2023-05-17 11:24:14,995 INFO akka.remote.Remoting [] - Starting remoting 2023-05-17 11:24:15,128 INFO akka.remote.Remoting [] - Remoting started; listening on addresses :[akka.tcp://flink@jykj2.yarn.com:38523] 2023-05-17 11:24:15,242 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor system started at akka.tcp://flink@jykj2.yarn.com:38523 2023-05-17 11:24:15,257 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Using working directory: WorkingDirectory(/hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/tm_container_1681371658709_0339_01_000002) 2023-05-17 11:24:15,267 INFO org.apache.flink.runtime.metrics.MetricRegistryImpl [] - No metrics reporter configured, no metrics will be exposed/reported. 2023-05-17 11:24:15,271 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Trying to start actor system, external address jykj2.yarn.com:0, bind address 0.0.0.0:0. 2023-05-17 11:24:15,286 INFO akka.event.slf4j.Slf4jLogger [] - Slf4jLogger started 2023-05-17 11:24:15,290 INFO akka.remote.RemoteActorRefProvider [] - Akka Cluster not in use - enabling unsafe features anyway becauseakka.remote.use-unsafe-remote-features-outside-clusterhas been enabled. 2023-05-17 11:24:15,290 INFO akka.remote.Remoting [] - Starting remoting 2023-05-17 11:24:15,297 INFO akka.remote.Remoting [] - Remoting started; listening on addresses :[akka.tcp://flink-metrics@jykj2.yarn.com:46016] 2023-05-17 11:24:15,306 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils [] - Actor system started at akka.tcp://flink-metrics@jykj2.yarn.com:46016 2023-05-17 11:24:15,320 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting RPC endpoint for org.apache.flink.runtime.metrics.dump.MetricQueryService at akka://flink-metrics/user/rpc/MetricQueryService_container_1681371658709_0339_01_000002 . 2023-05-17 11:24:15,332 INFO org.apache.flink.runtime.blob.PermanentBlobCache [] - Created BLOB cache storage directory /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/tm_container_1681371658709_0339_01_000002/blobStorage 2023-05-17 11:24:15,337 INFO org.apache.flink.runtime.blob.TransientBlobCache [] - Created BLOB cache storage directory /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/tm_container_1681371658709_0339_01_000002/blobStorage 2023-05-17 11:24:15,341 INFO org.apache.flink.runtime.externalresource.ExternalResourceUtils [] - Enabled external resources: [] 2023-05-17 11:24:15,341 INFO org.apache.flink.runtime.taskexecutor.TaskManagerRunner [] - Starting TaskManager with ResourceID: container_1681371658709_0339_01_000002(jykj2.yarn.com:36423) 2023-05-17 11:24:15,363 INFO org.apache.flink.runtime.taskexecutor.TaskManagerServices [] - Temporary file directory '/hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339': total 169 GB, usable 43 GB (25.44% usable) 2023-05-17 11:24:15,367 INFO org.apache.flink.runtime.io.disk.iomanager.IOManager [] - Created a new FileChannelManager for spilling of task related data to disk (joins, sorting, ...). Used directories: /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/flink-io-f132bfe4-6331-48da-b5ca-66b1db4c06b4 2023-05-17 11:24:15,375 INFO org.apache.flink.runtime.io.network.netty.NettyConfig [] - NettyConfig [server address: /0.0.0.0, server port: 0, ssl enabled: false, memory segment size (bytes): 32768, transport type: AUTO, number of server threads: 4 (manual), number of client threads: 4 (manual), server connect backlog: 0 (use Netty's default), client connect timeout (sec): 120, send/receive buffer size (bytes): 0 (use Netty's default)] 2023-05-17 11:24:15,443 INFO org.apache.flink.runtime.io.network.NettyShuffleServiceFactory [] - Created a new FileChannelManager for storing result partitions of BLOCKING shuffles. Used directories: /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/flink-netty-shuffle-93af0660-5454-4c07-a8a1-0c721d3e4ef1 2023-05-17 11:24:16,553 INFO org.apache.flink.runtime.io.network.buffer.NetworkBufferPool [] - Allocated 2048 MB for network buffer pool (number of memory segments: 65536, bytes per segment: 32768). 2023-05-17 11:24:16,567 INFO org.apache.flink.runtime.io.network.NettyShuffleEnvironment [] - Starting the network environment and its components. 2023-05-17 11:24:16,634 INFO org.apache.flink.runtime.io.network.netty.NettyClient [] - Transport type 'auto': using EPOLL. 2023-05-17 11:24:16,636 INFO org.apache.flink.runtime.io.network.netty.NettyClient [] - Successful initialization (took 68 ms). 2023-05-17 11:24:16,641 INFO org.apache.flink.runtime.io.network.netty.NettyServer [] - Transport type 'auto': using EPOLL. 2023-05-17 11:24:16,675 INFO org.apache.flink.runtime.io.network.netty.NettyServer [] - Successful initialization (took 37 ms). Listening on SocketAddress /0:0:0:0:0:0:0:0%0:44410. 2023-05-17 11:24:16,677 INFO org.apache.flink.runtime.taskexecutor.KvStateService [] - Starting the kvState service and its components. 2023-05-17 11:24:16,705 INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService [] - Starting RPC endpoint for org.apache.flink.runtime.taskexecutor.TaskExecutor at akka://flink/user/rpc/taskmanager_0 . 2023-05-17 11:24:16,731 INFO org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - Starting DefaultLeaderRetrievalService with ZookeeperLeaderRetrievalDriver{connectionInformationPath='/resource_manager/connection_info'}. 2023-05-17 11:24:16,732 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Start job leader service. 2023-05-17 11:24:16,734 INFO org.apache.flink.runtime.filecache.FileCache [] - User file cache uses directory /hadoop/hdfs/tmp/nm-local-dir/usercache/root/appcache/application_1681371658709_0339/flink-dist-cache-84bd56af-5981-437d-a9ef-4805140cf105 2023-05-17 11:24:16,740 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Connecting to ResourceManager akka.tcp://flink@jykj2.yarn.com:43740/user/rpc/resourcemanager_1(8de9487b6cfbadacad84bd9b513048dd). 2023-05-17 11:24:16,926 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Resolved ResourceManager address, beginning registration 2023-05-17 11:24:16,993 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Successful registration at resource manager akka.tcp://flink@jykj2.yarn.com:43740/user/rpc/resourcemanager_1 under registration id e330732fa87ed3f478e45e04090cf896. 2023-05-17 11:24:17,078 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Receive slot request bdbb3d425483c9e7214a772eaedf2ddd for job 40bf1281c6cf40497436c39d983c41a6 from resource manager with leader id 8de9487b6cfbadacad84bd9b513048dd. 2023-05-17 11:24:17,083 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Allocated slot for bdbb3d425483c9e7214a772eaedf2ddd. 2023-05-17 11:24:17,084 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Add job 40bf1281c6cf40497436c39d983c41a6 for job leader monitoring. 2023-05-17 11:24:17,086 INFO org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - Starting DefaultLeaderRetrievalService with ZookeeperLeaderRetrievalDriver{connectionInformationPath='/40bf1281c6cf40497436c39d983c41a6/connection_info'}. 2023-05-17 11:24:17,088 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Try to register at job manager akka.tcp://flink@jykj2.yarn.com:43740/user/rpc/jobmanager_2 with leader id ad84bd9b-5130-48dd-8de9-487b6cfbadac. 2023-05-17 11:24:17,103 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Resolved JobManager address, beginning registration 2023-05-17 11:24:17,121 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Successful registration at job manager akka.tcp://flink@jykj2.yarn.com:43740/user/rpc/jobmanager_2 for job 40bf1281c6cf40497436c39d983c41a6. 2023-05-17 11:24:17,122 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Establish JobManager connection for job 40bf1281c6cf40497436c39d983c41a6. 2023-05-17 11:24:17,124 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Offer reserved slots to the leader of job 40bf1281c6cf40497436c39d983c41a6. 2023-05-17 11:24:17,155 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Activate slot bdbb3d425483c9e7214a772eaedf2ddd. 2023-05-17 11:24:17,173 INFO org.apache.flink.runtime.state.changelog.StateChangelogStorageLoader [] - Creating a changelog storage with name 'memory'. 2023-05-17 11:24:17,334 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Received task Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0), deploy into slot with allocation id bdbb3d425483c9e7214a772eaedf2ddd. 2023-05-17 11:24:17,335 INFO org.apache.flink.runtime.taskmanager.Task [] - Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from CREATED to DEPLOYING. 2023-05-17 11:24:17,338 INFO org.apache.flink.runtime.taskmanager.Task [] - Loading JAR files for task Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0) [DEPLOYING]. 2023-05-17 11:24:17,339 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Activate slot bdbb3d425483c9e7214a772eaedf2ddd. 2023-05-17 11:24:17,838 INFO org.apache.flink.streaming.runtime.tasks.StreamTask [] - Using job/cluster config to configure application-defined state backend: EmbeddedRocksDBStateBackend{, localRocksDbDirectories=null, enableIncrementalCheckpointing=TRUE, numberOfTransferThreads=4, writeBatchSize=2097152} 2023-05-17 11:24:17,840 INFO org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend [] - Using predefined options: DEFAULT. 2023-05-17 11:24:17,840 INFO org.apache.flink.streaming.runtime.tasks.StreamTask [] - Using application-defined state backend: EmbeddedRocksDBStateBackend{, localRocksDbDirectories=null, enableIncrementalCheckpointing=TRUE, numberOfTransferThreads=4, writeBatchSize=2097152} 2023-05-17 11:24:17,840 INFO org.apache.flink.runtime.state.StateBackendLoader [] - State backend loader loads the state backend as EmbeddedRocksDBStateBackend 2023-05-17 11:24:17,845 INFO org.apache.flink.streaming.runtime.tasks.StreamTask [] - Using job/cluster config to configure application-defined checkpoint storage: org.apache.flink.runtime.state.storage.FileSystemCheckpointStorage@50d8ea2 2023-05-17 11:24:17,858 INFO org.apache.flink.runtime.taskmanager.Task [] - Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from DEPLOYING to INITIALIZING. 2023-05-17 11:24:18,286 WARN org.apache.flink.metrics.MetricGroup [] - Name collision: Group already contains a Metric with the name 'currentEmitEventTimeLag'. Metric will not be reported.[jykj2.yarn.com, taskmanager, container_1681371658709_0339_01_000002, insert-into_default_catalog.default_database.sink_ods, Source: source[1], 0] 2023-05-17 11:24:18,286 WARN org.apache.flink.metrics.MetricGroup [] - Name collision: Group already contains a Metric with the name 'sourceIdleTime'. Metric will not be reported.[jykj2.yarn.com, taskmanager, container_1681371658709_0339_01_000002, insert-into_default_catalog.default_database.sink_ods, Source: source[1], 0] 2023-05-17 11:24:18,324 INFO com.dtstack.chunjun.sink.DtOutputFormatSinkFunction [] - Start initialize output format state 2023-05-17 11:24:18,344 INFO org.apache.flink.api.java.typeutils.TypeExtractor [] - Field FormatState#state will be processed as GenericType. Please read the Flink documentation on "Data Types & Serialization" for details of the effect on performance and schema evolution. 2023-05-17 11:24:18,345 INFO org.apache.flink.api.java.typeutils.TypeExtractor [] - Field FormatState#metric will be processed as GenericType. Please read the Flink documentation on "Data Types & Serialization" for details of the effect on performance and schema evolution. 2023-05-17 11:24:18,364 INFO com.dtstack.chunjun.sink.DtOutputFormatSinkFunction [] - Is restored:false 2023-05-17 11:24:18,364 INFO com.dtstack.chunjun.sink.DtOutputFormatSinkFunction [] - End initialize output format state 2023-05-17 11:24:18,562 INFO com.dtstack.chunjun.connector.emqx.util.MqttConnectUtil [] - connect 1 times. 2023-05-17 11:24:18,905 INFO com.dtstack.chunjun.connector.emqx.util.MqttConnectUtil [] - emqx is connected = true 2023-05-17 11:24:18,984 INFO com.dtstack.chunjun.sink.format.BaseRichOutputFormat [] - [EmqxOutputFormat] open successfully, checkpointMode = EXACTLY_ONCE, checkpointEnabled = true, flushIntervalMills = 10000, batchSize = 1, [EmqxConfig]: { "semantic" : "at-least-once", "connectRetryTimes" : 10, "errorRecord" : 0, "checkFormat" : true, "parallelism" : 1, "executeDdlAble" : false, "errorPercentage" : -1, "flushIntervalMills" : 10000, "broker" : "tcp://10.10.2.34:1883", "fieldNameList" : [ ], "cleanSession" : true, "codec" : "plain", "password" : "******", "qos" : 2, "topic" : "cx", "batchSize" : 1, "speedBytes" : 0, "rowSizeCalculatorType" : "objectSizeCalculator", "username" : "admin" } 2023-05-17 11:24:18,994 INFO org.apache.flink.runtime.taskmanager.Task [] - Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from INITIALIZING to RUNNING. 2023-05-17 11:24:19,064 INFO com.ververica.cdc.connectors.mysql.source.reader.MySqlSourceReader [] - Add Split: MySqlSnapshotSplit{tableId=test.test, splitId='test.test:0', splitKeyType=[countINT NOT NULL], splitStart=null, splitEnd=null, highWatermark=null} 2023-05-17 11:24:19,064 INFO org.apache.flink.connector.base.source.reader.SourceReaderBase [] - Adding split(s) to reader: [MySqlSnapshotSplit{tableId=test.test, splitId='test.test:0', splitKeyType=[countINT NOT NULL], splitStart=null, splitEnd=null, highWatermark=null}] 2023-05-17 11:24:19,068 INFO org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher [] - Starting split fetcher 0 2023-05-17 11:24:19,077 WARN io.debezium.connector.mysql.MySqlConnection [] - Database configuration option 'serverTimezone' is set but is obsolete, please use 'connectionTimeZone' instead 2023-05-17 11:24:19,406 INFO com.ververica.cdc.connectors.mysql.debezium.task.context.StatefulTaskContext [] - Starting offset is initialized to {ts_sec=0, file=, pos=0, kind=EARLIEST, row=0, event=0} 2023-05-17 11:24:19,411 INFO io.debezium.relational.history.DatabaseHistoryMetrics [] - Started database history recovery 2023-05-17 11:24:19,412 INFO io.debezium.relational.history.DatabaseHistoryMetrics [] - Finished database history recovery of 0 change(s) in 1 ms 2023-05-17 11:24:19,451 INFO com.ververica.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask [] - Snapshot step 1 - Determining low watermark {ts_sec=0, file=mysql-bin.001936, pos=116483772, kind=SPECIFIC, gtids=06c63f88-860f-11e9-8d08-fa163ec7264f:1-930973921, d61acd6e-77e2-11e9-b09c-fa163e874df5:1-23933, row=0, event=0} for split MySqlSnapshotSplit{tableId=test.test, splitId='test.test:0', splitKeyType=[countINT NOT NULL], splitStart=null, splitEnd=null, highWatermark=null} 2023-05-17 11:24:19,454 INFO com.ververica.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask [] - Snapshot step 2 - Snapshotting data 2023-05-17 11:24:19,454 INFO com.ververica.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask [] - Exporting data from split 'test.test:0' of table test.test 2023-05-17 11:24:19,455 INFO com.ververica.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask [] - For split 'test.test:0' of table test.test using select statement: 'SELECT * FROMtest.test' 2023-05-17 11:24:19,474 INFO com.ververica.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask [] - Finished exporting 7 records for split 'test.test:0', total duration '00:00:00.02' 2023-05-17 11:24:19,475 INFO com.ververica.cdc.connectors.mysql.debezium.task.MySqlSnapshotSplitReadTask [] - Snapshot step 3 - Determining high watermark {ts_sec=0, file=mysql-bin.001936, pos=116483772, kind=SPECIFIC, gtids=06c63f88-860f-11e9-8d08-fa163ec7264f:1-930973921, d61acd6e-77e2-11e9-b09c-fa163e874df5:1-23933, row=0, event=0} for split MySqlSnapshotSplit{tableId=test.test, splitId='test.test:0', splitKeyType=[count` INT NOT NULL], splitStart=null, splitEnd=null, highWatermark=null} 2023-05-17 11:24:19,730 INFO com.dtstack.chunjun.sink.format.BaseRichOutputFormat [] - format state:null 2023-05-17 11:24:19,730 INFO com.dtstack.chunjun.sink.DtOutputFormatSinkFunction [] - OutputFormat format state:FormatState{numOfSubTask=0, state=null, metric={numWrite=LongCounter 0, conversionErrors=LongCounter 0, writeDuration=LongCounter 0, duplicateErrors=LongCounter 0, count=LongCounter 0, collectFailedCount=LongCounter 0, byteWrite=LongCounter 0, snapshotWrite=LongCounter 0, nullErrors=LongCounter 0, nErrors=LongCounter 0, otherErrors=LongCounter 0}, numberRead=0, numberWrite=0, jobId='null', fileIndex=-1} 2023-05-17 11:24:19,931 INFO org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher [] - Finished reading from splits [test.test:0] 2023-05-17 11:24:20,272 INFO com.dtstack.chunjun.sink.format.BaseRichOutputFormat [] - 异常是================com.dtstack.chunjun.throwable.WriteRecordException: java.lang.NullPointerException 2023-05-17 11:24:20,272 INFO com.dtstack.chunjun.sink.format.BaseRichOutputFormat [] - 异常的数据是=============:+I(2.2,1,hh) 2023-05-17 11:24:20,321 INFO com.dtstack.chunjun.sink.format.BaseRichOutputFormat [] - taskNumber[0] close() 2023-05-17 11:24:20,322 INFO com.dtstack.chunjun.dirty.utils.TablePrintUtil [] - +------------------+----------------------------------+-------------------------------------------------------+-------------------+--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+-------------------------+ | serialVersionUID | jobId | jobName | operatorName | dirtyContent | errorMessage | fieldName | createTime | +------------------+----------------------------------+-------------------------------------------------------+-------------------+--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+-------------------------+ | 1 | 40bf1281c6cf40497436c39d983c41a6 | insert-into_default_catalog.default_database.sink_ods | Sink: sink_ods[3] | {"arity":3,"rowKind":"INSERT"} | com.dtstack.chunjun.throwable.WriteRecordException: java.lang.NullPointerException at com.dtstack.chunjun.connector.emqx.sink.EmqxOutputFormat.writeSingleRecordInternal(EmqxOutputFormat.java:95) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:487) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:302) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:89) at com.dtstack.chunjun.sink.DtOutputFormatSinkFunction.invoke(DtOutputFormatSinkFunction.java:112) at org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:65) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56) at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29) at org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:39) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:313) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter$OutputCollector.collect(MySqlRecordEmitter.java:154) at com.ververica.cdc.debezium.table.RowDataDebeziumDeserializeSchema.emit(RowDataDebeziumDeserializeSchema.java:162) at com.ververica.cdc.debezium.table.RowDataDebeziumDeserializeSchema.deserialize(RowDataDebeziumDeserializeSchema.java:127) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitElement(MySqlRecordEmitter.java:129) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.processElement(MySqlRecordEmitter.java:111) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:83) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55) at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:143) at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385) at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at org.apache.flink.formats.json.JsonRowDataSerializationSchema.serialize(JsonRowDataSerializationSchema.java:99) at org.apache.flink.formats.json.JsonRowDataSerializationSchema.serialize(JsonRowDataSerializationSchema.java:43) at com.dtstack.chunjun.connector.emqx.converter.EmqxRowConverter.toExternal(EmqxRowConverter.java:57) at com.dtstack.chunjun.connector.emqx.converter.EmqxRowConverter.toExternal(EmqxRowConverter.java:32) at com.dtstack.chunjun.connector.emqx.sink.EmqxOutputFormat.writeSingleRecordInternal(EmqxOutputFormat.java:91) ... 37 more | null | 2023-05-17 11:24:20.274 | +------------------+----------------------------------+-------------------------------------------------------+-------------------+--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+-------------------------+

2023-05-17 11:24:40,328 INFO com.dtstack.chunjun.sink.format.BaseRichOutputFormat [] - subtask[0}] close() finished 2023-05-17 11:24:40,328 INFO org.apache.flink.connector.base.source.reader.SourceReaderBase [] - Closing Source Reader. 2023-05-17 11:24:40,329 INFO org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher [] - Shutting down split fetcher 0 2023-05-17 11:24:40,330 INFO com.ververica.cdc.connectors.mysql.source.reader.MySqlSplitReader [] - Close current debezium reader com.ververica.cdc.connectors.mysql.debezium.reader.SnapshotSplitReader 2023-05-17 11:24:40,341 INFO io.debezium.jdbc.JdbcConnection [] - Connection gracefully closed 2023-05-17 11:24:40,342 WARN com.ververica.cdc.connectors.mysql.debezium.reader.SnapshotSplitReader [] - Failed to close the snapshot split reader in 30 seconds. 2023-05-17 11:24:40,342 INFO org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher [] - Split fetcher 0 exited. 2023-05-17 11:24:40,345 WARN org.apache.flink.runtime.taskmanager.Task [] - Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from RUNNING to FAILED with failure cause: com.dtstack.chunjun.throwable.NoRestartException: The dirty consumer shutdown, due to the consumed count exceed the max-consumed [0] at com.dtstack.chunjun.dirty.consumer.DirtyDataCollector.addConsumed(DirtyDataCollector.java:99) at com.dtstack.chunjun.dirty.consumer.DirtyDataCollector.offer(DirtyDataCollector.java:73) at com.dtstack.chunjun.dirty.manager.DirtyManager.collect(DirtyManager.java:134) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeSingleRecord(BaseRichOutputFormat.java:492) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:302) at com.dtstack.chunjun.sink.format.BaseRichOutputFormat.writeRecord(BaseRichOutputFormat.java:89) at com.dtstack.chunjun.sink.DtOutputFormatSinkFunction.invoke(DtOutputFormatSinkFunction.java:112) at org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:65) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:56) at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:29) at org.apache.flink.streaming.api.operators.StreamFilter.processElement(StreamFilter.java:39) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:82) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:57) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:313) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter$OutputCollector.collect(MySqlRecordEmitter.java:154) at com.ververica.cdc.debezium.table.RowDataDebeziumDeserializeSchema.emit(RowDataDebeziumDeserializeSchema.java:162) at com.ververica.cdc.debezium.table.RowDataDebeziumDeserializeSchema.deserialize(RowDataDebeziumDeserializeSchema.java:127) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitElement(MySqlRecordEmitter.java:129) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.processElement(MySqlRecordEmitter.java:111) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:83) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55) at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:143) at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:385) at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:542) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:831) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:780) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:935) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:914) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:728) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:550) at java.lang.Thread.run(Thread.java:748)

2023-05-17 11:24:40,345 INFO org.apache.flink.runtime.taskmanager.Task [] - Freeing task resources for Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 (51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0). 2023-05-17 11:24:40,351 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Un-registering task and sending final execution state FAILED to JobManager for task Source: source[1] -> DropUpdateBefore[2] -> Sink: sink_ods[3] (1/1)#0 51da3cfabff2f40b3c8c018cad8f4360_cbc357ccb763df2852fee8c4fc7d55f2_0_0. 2023-05-17 11:24:40,772 INFO org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:0, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1, taskHeapMemory=1024.000mb (1073741824 bytes), taskOffHeapMemory=7.000mb (7340032 bytes), managedMemory=16.000mb (16777216 bytes), networkMemory=512.000mb (536870912 bytes)}, allocationId: bdbb3d425483c9e7214a772eaedf2ddd, jobId: 40bf1281c6cf40497436c39d983c41a6). 2023-05-17 11:24:40,774 INFO org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Remove job 40bf1281c6cf40497436c39d983c41a6 from job leader monitoring. 2023-05-17 11:24:40,775 INFO org.apache.flink.runtime.leaderretrieval.DefaultLeaderRetrievalService [] - Stopping DefaultLeaderRetrievalService. 2023-05-17 11:24:40,775 INFO org.apache.flink.runtime.leaderretrieval.ZooKeeperLeaderRetrievalDriver [] - Closing ZookeeperLeaderRetrievalDriver{connectionInformationPath='/40bf1281c6cf40497436c39d983c41a6/connection_info'}. 2023-05-17 11:24:40,781 INFO org.apache.flink.runtime.taskexecutor.TaskExecutor [] - Close JobManager connection for job 40bf1281c6cf40497436c39d983c41a6. `

20100507 commented 1 year ago

空指针这个比较明显；

DTStack / chunjun