search

DTStack / chunjun

A data integration framework
https://dtstack.github.io/chunjun/
Apache License 2.0
3.98k stars 1.69k forks source link

chunjun配置指标统计输出至mysql #1176

Open biandou1313 opened 2 years ago

biandou1313 commented 2 years ago

Search before asking

What happened

1、根据chunjun配置mysql 但是本地或者标准模式下 都无法找到mysqlreport

What you expected to happen

1、json中配置 { "job": { "content": [ { "reader": { "parameter": { "password": "123456", "dataSourceId": 8, "column": [ { "precision": 4, "name": "id", "columnDisplaySize": 4, "type": "INT" }, { "precision": 255, "name": "name", "columnDisplaySize": 255, "type": "VARCHAR" }, { "precision": 4, "name": "age", "columnDisplaySize": 4, "type": "TINYINT" }, { "precision": 255, "name": "phone", "columnDisplaySize": 255, "type": "VARCHAR" } ], "connection": [ { "jdbcUrl": [ "jdbc:mysql://172.18.8.77:3306/zk_test" ], "table": [ "one_copy1" ] } ], "increColumn":"id", "splitPk": "id", "username": "root" }, "name": "mysqlreader" }, "writer": { "parameter": { "password": "123456", "dataSourceId": 8, "column": [ { "precision": 4, "name": "id", "columnDisplaySize": 4, "type": "INT" }, { "precision": 255, "name": "name", "columnDisplaySize": 255, "type": "VARCHAR" }, { "precision": 4, "name": "age", "columnDisplaySize": 4, "type": "TINYINT" }, { "precision": 255, "name": "phone", "columnDisplaySize": 255, "type": "VARCHAR" } ], "connection": [ { "jdbcUrl": "jdbc:mysql://172.18.8.77:3306/zk_test", "table": [ "one_copy1" ] } ], "writeMode": "insert", "username": "root" }, "name": "mysqlwriter" } } ], "setting": { "log": { "isLogger": false }, "errorLimit" : { "record": 100, "percentage": 100 }, "metricPluginConf" : { "pluginName": "promethus", "pluginProp": { "jdbcUrl":"jdbc:mysql://172.18.8.77:3306/zk_test", "database":"zk_test", "table":"test001", "username":"root", "password":"123456", "properties":{ } } }, "speed": { "bytes": 0, "channel": 1 } } } }

How to reproduce

{ "job": { "content": [ { "reader": { "parameter": { "password": "123456", "dataSourceId": 8, "column": [ { "precision": 4, "name": "id", "columnDisplaySize": 4, "type": "INT" }, { "precision": 255, "name": "name", "columnDisplaySize": 255, "type": "VARCHAR" }, { "precision": 4, "name": "age", "columnDisplaySize": 4, "type": "TINYINT" }, { "precision": 255, "name": "phone", "columnDisplaySize": 255, "type": "VARCHAR" } ], "connection": [ { "jdbcUrl": [ "jdbc:mysql://172.18.8.77:3306/zk_test" ], "table": [ "one_copy1" ] } ], "increColumn":"id", "splitPk": "id", "username": "root" }, "name": "mysqlreader" }, "writer": { "parameter": { "password": "123456", "dataSourceId": 8, "column": [ { "precision": 4, "name": "id", "columnDisplaySize": 4, "type": "INT" }, { "precision": 255, "name": "name", "columnDisplaySize": 255, "type": "VARCHAR" }, { "precision": 4, "name": "age", "columnDisplaySize": 4, "type": "TINYINT" }, { "precision": 255, "name": "phone", "columnDisplaySize": 255, "type": "VARCHAR" } ], "connection": [ { "jdbcUrl": "jdbc:mysql://172.18.8.77:3306/zk_test", "table": [ "one_copy1" ] } ], "writeMode": "insert", "username": "root" }, "name": "mysqlwriter" } } ], "setting": { "log": { "isLogger": false }, "errorLimit" : { "record": 100, "percentage": 100 }, "metricPluginConf" : { "pluginName": "promethus", "pluginProp": { "jdbcUrl":"jdbc:mysql://172.18.8.77:3306/zk_test", "database":"zk_test", "table":"test001", "username":"root", "password":"123456", "properties":{ } } }, "speed": { "bytes": 0, "channel": 1 } } } }

Anything else

No response

Version

master

Are you willing to submit PR?

Code of Conduct

biandou1313 commented 2 years ago

C:\develop\jdk8\bin\java.exe -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:55801,suspend=y,server=n -javaagent:C:\Users\admin\AppData\Local\JetBrains\IntelliJIdea2020.3\captureAgent\debugger-agent.jar -Dfile.encoding=UTF-8 -classpath C:\Users\admin\AppData\Local\Temp\classpath1169876339.jar com.dtstack.chunjun.local.test.LocalTest Connected to the target VM, address: '127.0.0.1:55801', transport: 'socket' 2022-08-23 15:45:59,516 - 0 WARN [main] com.dtstack.chunjun.local.test.LocalTest:----- 2022-08-23 15:45:59,760 - 244 INFO [main] com.dtstack.chunjun.Main:------------program params------------------------- 2022-08-23 15:45:59,839 - 323 INFO [main] com.dtstack.chunjun.Main:-mode 2022-08-23 15:45:59,840 - 324 INFO [main] com.dtstack.chunjun.Main:local 2022-08-23 15:45:59,842 - 326 INFO [main] com.dtstack.chunjun.Main:-jobType 2022-08-23 15:45:59,842 - 326 INFO [main] com.dtstack.chunjun.Main:sync 2022-08-23 15:45:59,842 - 326 INFO [main] com.dtstack.chunjun.Main:-job 2022-08-23 15:45:59,842 - 326 INFO [main] com.dtstack.chunjun.Main:%7B%0A++%22job%22%3A+%7B%0A++++%22content%22%3A+%5B%0A++++++%7B%0A++++++++%22reader%22%3A+%7B%0A++++++++++%22parameter%22%3A+%7B%0A++++++++++++%22password%22%3A+%22123456%22%2C%0A++++++++++++%22dataSourceId%22%3A+8%2C%0A++++++++++++%22column%22%3A+%5B%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+4%2C%0A++++++++++++++++%22name%22%3A+%22id%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+4%2C%0A++++++++++++++++%22type%22%3A+%22INT%22%0A++++++++++++++%7D%2C%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+255%2C%0A++++++++++++++++%22name%22%3A+%22name%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+255%2C%0A++++++++++++++++%22type%22%3A+%22VARCHAR%22%0A++++++++++++++%7D%2C%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+4%2C%0A++++++++++++++++%22name%22%3A+%22age%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+4%2C%0A++++++++++++++++%22type%22%3A+%22TINYINT%22%0A++++++++++++++%7D%2C%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+255%2C%0A++++++++++++++++%22name%22%3A+%22phone%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+255%2C%0A++++++++++++++++%22type%22%3A+%22VARCHAR%22%0A++++++++++++++%7D%0A++++++++++++%5D%2C%0A++++++++++++%22connection%22%3A+%5B%0A++++++++++++++%7B%0A++++++++++++++++%22jdbcUrl%22%3A+%5B%0A++++++++++++++++++%22jdbc%3Amysql%3A%2F%2F172.18.8.77%3A3306%2Fzk_test%22%0A++++++++++++++++%5D%2C%0A++++++++++++++++%22table%22%3A+%5B%0A++++++++++++++++++%22one%22%0A++++++++++++++++%5D%0A++++++++++++++%7D%0A++++++++++++%5D%2C%0A++++++++++++%22increColumn%22%3A%22id%22%2C%0A++++++++++++%22splitPk%22%3A+%22id%22%2C%0A++++++++++++%22username%22%3A+%22root%22%0A++++++++++%7D%2C%0A++++++++++%22name%22%3A+%22mysqlreader%22%0A++++++++%7D%2C%0A++++++++%22writer%22%3A+%7B%0A++++++++++%22parameter%22%3A+%7B%0A++++++++++++%22password%22%3A+%22123456%22%2C%0A++++++++++++%22dataSourceId%22%3A+8%2C%0A++++++++++++%22column%22%3A+%5B%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+4%2C%0A++++++++++++++++%22name%22%3A+%22id%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+4%2C%0A++++++++++++++++%22type%22%3A+%22INT%22%0A++++++++++++++%7D%2C%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+255%2C%0A++++++++++++++++%22name%22%3A+%22name%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+255%2C%0A++++++++++++++++%22type%22%3A+%22VARCHAR%22%0A++++++++++++++%7D%2C%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+4%2C%0A++++++++++++++++%22name%22%3A+%22age%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+4%2C%0A++++++++++++++++%22type%22%3A+%22TINYINT%22%0A++++++++++++++%7D%2C%0A++++++++++++++%7B%0A++++++++++++++++%22precision%22%3A+255%2C%0A++++++++++++++++%22name%22%3A+%22phone%22%2C%0A++++++++++++++++%22columnDisplaySize%22%3A+255%2C%0A++++++++++++++++%22type%22%3A+%22VARCHAR%22%0A++++++++++++++%7D%0A++++++++++++%5D%2C%0A++++++++++++%22connection%22%3A+%5B%0A++++++++++++++%7B%0A++++++++++++++++%22jdbcUrl%22%3A+%22jdbc%3Amysql%3A%2F%2F172.18.8.77%3A3306%2Fzk_test%22%2C%0A++++++++++++++++%22table%22%3A+%5B%0A++++++++++++++++++%22one_copy1%22%0A++++++++++++++++%5D%0A++++++++++++++%7D%0A++++++++++++%5D%2C%0A++++++++++++%22writeMode%22%3A+%22insert%22%2C%0A++++++++++++%22username%22%3A+%22root%22%0A++++++++++%7D%2C%0A++++++++++%22name%22%3A+%22mysqlwriter%22%0A++++++++%7D%0A++++++%7D%0A++++%5D%2C%0A++++%22setting%22%3A+%7B%0A++++++%22log%22%3A+%7B%0A++++++++%22isLogger%22%3A+false%0A++++++%7D%2C%0A++++++%22errorLimit%22+%3A+%7B%0A++++++++%22record%22%3A+100%2C%0A++++++++%22percentage%22%3A+100%0A++++++%7D%2C%0A++++++%22metricPluginConf%22+%3A+%7B%0A++++++++%22pluginName%22%3A+%22mysql%22%2C%0A++++++++%22pluginProp%22%3A+%7B%0A++++++++++%22jdbcUrl%22%3A%22jdbc%3Amysql%3A%2F%2F172.18.8.77%3A3306%2Fzk_test%22%2C%0A++++++++++%22database%22%3A%22zk_test%22%2C%0A++++++++++%22table%22%3A%22test001%22%2C%0A++++++++++%22username%22%3A%22root%22%2C%0A++++++++++%22password%22%3A%22123456%22%2C%0A++++++++++%22properties%22%3A%7B%0A++++++++++%7D%0A++++++++%7D%0A++++++%7D%2C%0A++++++%22speed%22%3A+%7B%0A++++++++%22bytes%22%3A+0%2C%0A++++++++%22channel%22%3A+1%0A++++++%7D%0A++++%7D%0A++%7D%0A%7D%0A 2022-08-23 15:45:59,842 - 326 INFO [main] com.dtstack.chunjun.Main:{} 2022-08-23 15:45:59,842 - 326 INFO [main] com.dtstack.chunjun.Main:------------------------------------------- 2022-08-23 15:46:02,387 - 2871 INFO [main] com.dtstack.chunjun.Main:Register to table configuration:{table.dynamic-table-options.enabled=true, pipeline.name=Flink_Job} 2022-08-23 15:46:02,436 - 2920 INFO [main] com.dtstack.chunjun.util.PluginUtil:ChunJun executionMode: local 2022-08-23 15:46:02,437 - 2921 INFO [main] com.dtstack.chunjun.util.PluginUtil:ChunJun reset pipeline.jars: [] 2022-08-23 15:46:02,445 - 2929 INFO [main] com.dtstack.chunjun.classloader.ClassLoaderManager:jarUrl: create ClassLoad successful... 2022-08-23 15:46:09,035 - 9519 INFO [main] org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils:The configuration option taskmanager.cpu.cores required for local execution is not set, setting it to the maximal possible value. 2022-08-23 15:46:09,036 - 9520 INFO [main] org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils:The configuration option taskmanager.memory.task.heap.size required for local execution is not set, setting it to the maximal possible value. 2022-08-23 15:46:09,039 - 9523 INFO [main] org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils:The configuration option taskmanager.memory.task.off-heap.size required for local execution is not set, setting it to the maximal possible value. 2022-08-23 15:46:09,043 - 9527 INFO [main] org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils:The configuration option taskmanager.memory.network.min required for local execution is not set, setting it to its default value 64 mb. 2022-08-23 15:46:09,043 - 9527 INFO [main] org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils:The configuration option taskmanager.memory.network.max required for local execution is not set, setting it to its default value 64 mb. 2022-08-23 15:46:09,043 - 9527 INFO [main] com.dtstack.chunjun.environment.MyLocalStreamEnvironment:Running job on local embedded Flink mini cluster 2022-08-23 15:46:09,262 - 9746 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Starting Flink Mini Cluster 2022-08-23 15:46:09,267 - 9751 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Starting Metrics Registry 2022-08-23 15:46:09,421 - 9905 INFO [main] org.apache.flink.runtime.metrics.MetricRegistryImpl:No metrics reporter configured, no metrics will be exposed/reported. 2022-08-23 15:46:09,422 - 9906 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Starting RPC Service(s) 2022-08-23 15:46:09,465 - 9949 INFO [main] org.apache.flink.runtime.clusterframework.BootstrapTools:Trying to start local actor system 2022-08-23 15:46:10,839 - 11323 INFO [flink-akka.actor.default-dispatcher-2] akka.event.slf4j.Slf4jLogger$$anonfun$receive$1:Slf4jLogger started 2022-08-23 15:46:12,689 - 13173 INFO [main] org.apache.flink.runtime.clusterframework.BootstrapTools:Actor system started at akka://flink 2022-08-23 15:46:12,725 - 13209 INFO [main] org.apache.flink.runtime.clusterframework.BootstrapTools:Trying to start local actor system 2022-08-23 15:46:12,790 - 13274 INFO [flink-metrics-2] akka.event.slf4j.Slf4jLogger$$anonfun$receive$1:Slf4jLogger started 2022-08-23 15:46:13,552 - 14036 INFO [main] org.apache.flink.runtime.clusterframework.BootstrapTools:Actor system started at akka://flink-metrics 2022-08-23 15:46:13,585 - 14069 INFO [main] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Starting RPC endpoint for org.apache.flink.runtime.metrics.dump.MetricQueryService at akka://flink-metrics/user/rpc/MetricQueryService . 2022-08-23 15:46:13,721 - 14205 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Starting high-availability services 2022-08-23 15:46:13,826 - 14310 INFO [main] org.apache.flink.runtime.blob.BlobServer:Created BLOB server storage directory C:\Users\admin\AppData\Local\Temp\blobStore-e510febb-b3f0-4f8c-b891-c411a0caeefd 2022-08-23 15:46:13,848 - 14332 INFO [main] org.apache.flink.runtime.blob.BlobServer:Started BLOB server at 0.0.0.0:55809 - max concurrent requests: 50 - max backlog: 1000 2022-08-23 15:46:13,859 - 14343 INFO [main] org.apache.flink.runtime.blob.AbstractBlobCache:Created BLOB cache storage directory C:\Users\admin\AppData\Local\Temp\blobStore-26c7b8f7-ec47-4f1d-bb32-ff3820398b72 2022-08-23 15:46:13,862 - 14346 INFO [main] org.apache.flink.runtime.blob.AbstractBlobCache:Created BLOB cache storage directory C:\Users\admin\AppData\Local\Temp\blobStore-5a6877e6-6c75-4d68-b135-0a4d73d5527d 2022-08-23 15:46:13,862 - 14346 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Starting 1 TaskManger(s) 2022-08-23 15:46:13,872 - 14356 INFO [main] org.apache.flink.runtime.taskexecutor.TaskManagerRunner:Starting TaskManager with ResourceID: b5e95074-82cb-4551-aaf6-3327da83c63b 2022-08-23 15:46:13,943 - 14427 INFO [main] org.apache.flink.runtime.taskexecutor.TaskManagerServices:Temporary file directory 'C:\Users\admin\AppData\Local\Temp': total 476 GB, usable 344 GB (72.27% usable) 2022-08-23 15:46:13,956 - 14440 INFO [main] org.apache.flink.runtime.io.disk.FileChannelManagerImpl:FileChannelManager uses directory C:\Users\admin\AppData\Local\Temp\flink-io-5658fc3b-33b8-485b-84c4-753623e90653 for spill files. 2022-08-23 15:46:13,985 - 14469 INFO [main] org.apache.flink.runtime.io.disk.FileChannelManagerImpl:FileChannelManager uses directory C:\Users\admin\AppData\Local\Temp\flink-netty-shuffle-caf0f9b6-89b8-4020-85d7-f97670524c38 for spill files. 2022-08-23 15:46:14,042 - 14526 INFO [main] org.apache.flink.runtime.io.network.buffer.NetworkBufferPool:Allocated 64 MB for network buffer pool (number of memory segments: 2048, bytes per segment: 32768). 2022-08-23 15:46:14,090 - 14574 INFO [main] org.apache.flink.runtime.io.network.NettyShuffleEnvironment:Starting the network environment and its components. 2022-08-23 15:46:14,096 - 14580 INFO [main] org.apache.flink.runtime.taskexecutor.KvStateService:Starting the kvState service and its components. 2022-08-23 15:46:14,175 - 14659 INFO [main] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Starting RPC endpoint for org.apache.flink.runtime.taskexecutor.TaskExecutor at akka://flink/user/rpc/taskmanager_0 . 2022-08-23 15:46:14,220 - 14704 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService:Start job leader service. 2022-08-23 15:46:14,221 - 14705 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.filecache.FileCache:User file cache uses directory C:\Users\admin\AppData\Local\Temp\flink-dist-cache-b7cf9ffc-6e03-46c7-b797-97c0ca8e3f84 2022-08-23 15:46:14,360 - 14844 INFO [main] org.apache.flink.runtime.rest.RestServerEndpoint:Starting rest endpoint. 2022-08-23 15:46:15,022 - 15506 WARN [main] org.apache.flink.runtime.webmonitor.WebMonitorUtils$LogFileLocation:Log file environment variable 'log.file' is not set. 2022-08-23 15:46:15,022 - 15506 WARN [main] org.apache.flink.runtime.webmonitor.WebMonitorUtils$LogFileLocation:JobManager log files are unavailable in the web dashboard. Log file location not found in environment variable 'log.file' or configuration key 'web.log.path'. 2022-08-23 15:46:15,571 - 16055 INFO [main] org.apache.flink.runtime.rest.RestServerEndpoint:Rest endpoint listening at localhost:55877 2022-08-23 15:46:15,572 - 16056 INFO [main] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Proposing leadership to contender http://localhost:55877 2022-08-23 15:46:15,580 - 16064 INFO [main] org.apache.flink.runtime.webmonitor.WebMonitorEndpoint:Web frontend listening at http://localhost:55877. 2022-08-23 15:46:15,581 - 16065 INFO [mini-cluster-io-thread-1] org.apache.flink.runtime.webmonitor.WebMonitorEndpoint:http://localhost:55877 was granted leadership with leaderSessionID=73976242-202e-4150-9ed9-7138d3cf094d 2022-08-23 15:46:15,581 - 16065 INFO [mini-cluster-io-thread-1] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Received confirmation of leadership for leader http://localhost:55877 , session=73976242-202e-4150-9ed9-7138d3cf094d 2022-08-23 15:46:15,623 - 16107 INFO [main] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Starting RPC endpoint for org.apache.flink.runtime.resourcemanager.StandaloneResourceManager at akka://flink/user/rpc/resourcemanager_1 . 2022-08-23 15:46:15,688 - 16172 INFO [main] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Proposing leadership to contender LeaderContender: DefaultDispatcherRunner 2022-08-23 15:46:15,689 - 16173 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Proposing leadership to contender LeaderContender: StandaloneResourceManager 2022-08-23 15:46:15,691 - 16175 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.resourcemanager.ResourceManager:ResourceManager akka://flink/user/rpc/resourcemanager_1 was granted leadership with fencing token b2fe742345784dcd9ca6bd679e5d46fb 2022-08-23 15:46:15,694 - 16178 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl:Starting the SlotManager. 2022-08-23 15:46:15,695 - 16179 INFO [mini-cluster-io-thread-2] org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess:Start SessionDispatcherLeaderProcess. 2022-08-23 15:46:15,696 - 16180 INFO [mini-cluster-io-thread-4] org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess:Recover all persisted job graphs. 2022-08-23 15:46:15,697 - 16181 INFO [mini-cluster-io-thread-4] org.apache.flink.runtime.dispatcher.runner.SessionDispatcherLeaderProcess:Successfully recovered 0 persisted job graphs. 2022-08-23 15:46:15,698 - 16182 INFO [mini-cluster-io-thread-5] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Received confirmation of leadership for leader akka://flink/user/rpc/resourcemanager_1 , session=9ca6bd67-9e5d-46fb-b2fe-742345784dcd 2022-08-23 15:46:15,700 - 16184 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Flink Mini Cluster started successfully 2022-08-23 15:46:15,700 - 16184 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.TaskExecutor:Connecting to ResourceManager akka://flink/user/rpc/resourcemanager_1(b2fe742345784dcd9ca6bd679e5d46fb). 2022-08-23 15:46:15,704 - 16188 INFO [mini-cluster-io-thread-4] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Starting RPC endpoint for org.apache.flink.runtime.dispatcher.StandaloneDispatcher at akka://flink/user/rpc/dispatcher_2 . 2022-08-23 15:46:15,714 - 16198 INFO [mini-cluster-io-thread-4] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Received confirmation of leadership for leader akka://flink/user/rpc/dispatcher_2 , session=68738c52-9acf-4a6f-9283-7ecefee4f057 2022-08-23 15:46:15,726 - 16210 INFO [flink-akka.actor.default-dispatcher-3] org.apache.flink.runtime.registration.RetryingRegistration:Resolved ResourceManager address, beginning registration 2022-08-23 15:46:15,735 - 16219 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.resourcemanager.ResourceManager:Registering TaskManager with ResourceID b5e95074-82cb-4551-aaf6-3327da83c63b (akka://flink/user/rpc/taskmanager_0) at ResourceManager 2022-08-23 15:46:15,738 - 16222 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.TaskExecutorToResourceManagerConnection:Successful registration at resource manager akka://flink/user/rpc/resourcemanager_1 under registration id 9166c2ba294736ee2efd8e89b73454d8. 2022-08-23 15:46:15,741 - 16225 INFO [flink-akka.actor.default-dispatcher-3] org.apache.flink.runtime.dispatcher.Dispatcher:Received JobGraph submission 235c680366272aa60c6c996eaf429555 (Flink_Job). 2022-08-23 15:46:15,742 - 16226 INFO [flink-akka.actor.default-dispatcher-3] org.apache.flink.runtime.dispatcher.Dispatcher:Submitting job 235c680366272aa60c6c996eaf429555 (Flink_Job). 2022-08-23 15:46:15,767 - 16251 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Proposing leadership to contender LeaderContender: JobManagerRunnerImpl 2022-08-23 15:46:15,777 - 16261 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/rpc/jobmanager_3 . 2022-08-23 15:46:15,833 - 16317 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.jobmaster.JobMaster:Initializing job Flink_Job (235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:15,874 - 16358 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.scheduler.DefaultSchedulerFactory:Using restart back off time strategy NoRestartBackoffTimeStrategy for Flink_Job (235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:15,906 - 16390 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder:start to buildGraph for job Flink_Job (235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:15,938 - 16422 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder:trying to download shipFile from blobServer for job Flink_Job (235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:15,939 - 16423 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder:Running initialization on master for job Flink_Job (235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:15,944 - 16428 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder:Successfully ran initialization on master in 5 ms. 2022-08-23 15:46:15,957 - 16441 INFO [mini-cluster-io-thread-12] com.dtstack.chunjun.connector.jdbc.source.JdbcInputFormat:createInputSplitsInternal success, splits is [ { "mod": 0, "isPolling": false, "splitStrategy": "mod", "rangeEndLocationOperator": " < ", "partitionNumber": 0, "totalNumberOfPartitions": 1 } ] 2022-08-23 15:46:15,964 - 16448 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.scheduler.adapter.DefaultExecutionTopology:Built 1 pipelined regions in 0 ms 2022-08-23 15:46:15,978 - 16462 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.state.StateBackendLoader:No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize: 5242880) 2022-08-23 15:46:15,994 - 16478 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.checkpoint.CheckpointCoordinator:No checkpoint found during restore. 2022-08-23 15:46:16,002 - 16486 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.scheduler.DefaultScheduler:Using failover strategy org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@4ae23288 for Flink_Job (235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:16,019 - 16503 INFO [mini-cluster-io-thread-12] org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl:JobManager runner for job Flink_Job (235c680366272aa60c6c996eaf429555) was granted leadership with session id 38ab2685-521d-4540-9be4-ee45fbf09998 at akka://flink/user/rpc/jobmanager_3. 2022-08-23 15:46:16,025 - 16509 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.jobmaster.JobMaster:Starting execution of job Flink_Job (235c680366272aa60c6c996eaf429555) under job master id 9be4ee45fbf0999838ab2685521d4540. 2022-08-23 15:46:16,028 - 16512 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.scheduler.DefaultScheduler:Starting scheduling with scheduling strategy [org.apache.flink.runtime.scheduler.strategy.PipelinedRegionSchedulingStrategy] 2022-08-23 15:46:16,030 - 16514 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.executiongraph.ExecutionGraph:Job Flink_Job (235c680366272aa60c6c996eaf429555) switched from state CREATED to RUNNING. 2022-08-23 15:46:16,034 - 16518 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.executiongraph.Execution:Source: mysqlsourcefactory (1/1) (4ec78812b87fd6bff86fd1466c7a4257) switched from CREATED to SCHEDULED. 2022-08-23 15:46:16,034 - 16518 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.executiongraph.Execution:Sink: mysqlsinkfactory (1/1) (85c435b87850c0169bdfc868b25718e3) switched from CREATED to SCHEDULED. 2022-08-23 15:46:16,044 - 16528 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl:Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{3a4c5751a3b9db31bc8d80b03a29aacf}] 2022-08-23 15:46:16,069 - 16553 INFO [jobmanager-future-thread-1] org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService:Received confirmation of leadership for leader akka://flink/user/rpc/jobmanager_3 , session=38ab2685-521d-4540-9be4-ee45fbf09998 2022-08-23 15:46:16,069 - 16553 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.jobmaster.JobMaster:Connecting to ResourceManager akka://flink/user/rpc/resourcemanager_1(b2fe742345784dcd9ca6bd679e5d46fb) 2022-08-23 15:46:16,071 - 16555 INFO [flink-akka.actor.default-dispatcher-3] org.apache.flink.runtime.registration.RetryingRegistration:Resolved ResourceManager address, beginning registration 2022-08-23 15:46:16,073 - 16557 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.resourcemanager.ResourceManager:Registering job manager 9be4ee45fbf0999838ab2685521d4540@akka://flink/user/rpc/jobmanager_3 for job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:16,083 - 16567 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.resourcemanager.ResourceManager:Registered job manager 9be4ee45fbf0999838ab2685521d4540@akka://flink/user/rpc/jobmanager_3 for job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:16,094 - 16578 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.jobmaster.JobMaster:JobManager successfully registered at ResourceManager, leader id: b2fe742345784dcd9ca6bd679e5d46fb. 2022-08-23 15:46:16,097 - 16581 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl:Requesting new slot [SlotRequestId{3a4c5751a3b9db31bc8d80b03a29aacf}] and profile ResourceProfile{UNKNOWN} with allocation id 2df6d256a754041030bc7a2fc03cbcb9 from resource manager. 2022-08-23 15:46:16,099 - 16583 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.resourcemanager.ResourceManager:Request slot with profile ResourceProfile{UNKNOWN} for job 235c680366272aa60c6c996eaf429555 with allocation id 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,102 - 16586 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.taskexecutor.TaskExecutor:Receive slot request 2df6d256a754041030bc7a2fc03cbcb9 for job 235c680366272aa60c6c996eaf429555 from resource manager with leader id b2fe742345784dcd9ca6bd679e5d46fb. 2022-08-23 15:46:16,110 - 16594 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.taskexecutor.TaskExecutor:Allocated slot for 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,112 - 16596 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService:Add job 235c680366272aa60c6c996eaf429555 for job leader monitoring. 2022-08-23 15:46:16,125 - 16609 INFO [mini-cluster-io-thread-18] org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService$JobManagerLeaderListener:Try to register at job manager akka://flink/user/rpc/jobmanager_3 with leader id 38ab2685-521d-4540-9be4-ee45fbf09998. 2022-08-23 15:46:16,126 - 16610 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.registration.RetryingRegistration:Resolved JobManager address, beginning registration 2022-08-23 15:46:16,132 - 16616 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService$JobManagerLeaderListener$JobManagerRegisteredRpcConnection:Successful registration at job manager akka://flink/user/rpc/jobmanager_3 for job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:16,133 - 16617 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.TaskExecutor:Establish JobManager connection for job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:16,138 - 16622 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.TaskExecutor:Offer reserved slots to the leader of job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:16,146 - 16630 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.Execution:Source: mysqlsourcefactory (1/1) (4ec78812b87fd6bff86fd1466c7a4257) switched from SCHEDULED to DEPLOYING. 2022-08-23 15:46:16,150 - 16634 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.Execution:Deploying Source: mysqlsourcefactory (1/1) (attempt #0) with attempt id 4ec78812b87fd6bff86fd1466c7a4257 to b5e95074-82cb-4551-aaf6-3327da83c63b @ 127.0.0.1 (dataPort=-1) with allocation id 2df6d256a754041030bc7a2fc03cbcb9 2022-08-23 15:46:16,155 - 16639 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.Execution:Sink: mysqlsinkfactory (1/1) (85c435b87850c0169bdfc868b25718e3) switched from SCHEDULED to DEPLOYING. 2022-08-23 15:46:16,156 - 16640 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.Execution:Deploying Sink: mysqlsinkfactory (1/1) (attempt #0) with attempt id 85c435b87850c0169bdfc868b25718e3 to b5e95074-82cb-4551-aaf6-3327da83c63b @ 127.0.0.1 (dataPort=-1) with allocation id 2df6d256a754041030bc7a2fc03cbcb9 2022-08-23 15:46:16,156 - 16640 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl:Activate slot 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,193 - 16677 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.TaskExecutor:Received task Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257), deploy into slot with allocation id 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,195 - 16679 INFO [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257) switched from CREATED to DEPLOYING. 2022-08-23 15:46:16,196 - 16680 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl:Activate slot 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,202 - 16686 INFO [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Loading JAR files for task Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257) [DEPLOYING]. 2022-08-23 15:46:16,203 - 16687 INFO [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Registering task at network: Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257) [DEPLOYING]. 2022-08-23 15:46:16,214 - 16698 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.TaskExecutor:Received task Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3), deploy into slot with allocation id 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,215 - 16699 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3) switched from CREATED to DEPLOYING. 2022-08-23 15:46:16,216 - 16700 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl:Activate slot 2df6d256a754041030bc7a2fc03cbcb9. 2022-08-23 15:46:16,218 - 16702 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Loading JAR files for task Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3) [DEPLOYING]. 2022-08-23 15:46:16,221 - 16705 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Registering task at network: Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3) [DEPLOYING]. 2022-08-23 15:46:16,243 - 16727 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.state.StateBackendLoader:No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize: 5242880) 2022-08-23 15:46:16,244 - 16728 INFO [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.state.StateBackendLoader:No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize: 5242880) 2022-08-23 15:46:16,262 - 16746 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3) switched from DEPLOYING to RUNNING. 2022-08-23 15:46:16,262 - 16746 INFO [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257) switched from DEPLOYING to RUNNING. 2022-08-23 15:46:16,265 - 16749 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.executiongraph.Execution:Sink: mysqlsinkfactory (1/1) (85c435b87850c0169bdfc868b25718e3) switched from DEPLOYING to RUNNING. 2022-08-23 15:46:16,266 - 16750 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.executiongraph.Execution:Source: mysqlsourcefactory (1/1) (4ec78812b87fd6bff86fd1466c7a4257) switched from DEPLOYING to RUNNING. 2022-08-23 15:46:16,339 - 16823 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.DtOutputFormatSinkFunction:Start initialize output format state 2022-08-23 15:46:16,339 - 16823 INFO [Source: mysqlsourcefactory (1/1)#0] com.dtstack.chunjun.source.DtInputFormatSourceFunction:Start initialize input format state, is restored:false 2022-08-23 15:46:16,396 - 16880 INFO [Source: mysqlsourcefactory (1/1)#0] com.dtstack.chunjun.source.DtInputFormatSourceFunction:End initialize input format state 2022-08-23 15:46:16,396 - 16880 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.DtOutputFormatSinkFunction:Is restored:false 2022-08-23 15:46:16,397 - 16881 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.DtOutputFormatSinkFunction:End initialize output format state 2022-08-23 15:46:16,398 - 16882 WARN [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.taskmanager.TaskManagerLocation:No hostname could be resolved for the IP address 127.0.0.1, using IP address as host name. Local input split assignment (such as for HDFS files) may be impacted. 2022-08-23 15:46:16,411 - 16895 ERROR [Legacy Source Thread - Source: mysqlsourcefactory (1/1)#0] com.dtstack.chunjun.source.DtInputFormatSourceFunction:Exception happened, start to close format com.dtstack.chunjun.throwable.ChunJunRuntimeException: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:111) at com.dtstack.chunjun.source.format.BaseRichInputFormat.openInputFormat(BaseRichInputFormat.java:182) at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:106) ... 5 more 2022-08-23 15:46:16,413 - 16897 INFO [Legacy Source Thread - Source: mysqlsourcefactory (1/1)#0] com.dtstack.chunjun.source.format.BaseRichInputFormat:subtask input close finished 2022-08-23 15:46:16,418 - 16902 WARN [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257) switched from RUNNING to FAILED. com.dtstack.chunjun.throwable.ChunJunRuntimeException: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:111) at com.dtstack.chunjun.source.format.BaseRichInputFormat.openInputFormat(BaseRichInputFormat.java:182) at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:106) ... 5 more 2022-08-23 15:46:16,420 - 16904 INFO [Source: mysqlsourcefactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Freeing task resources for Source: mysqlsourcefactory (1/1)#0 (4ec78812b87fd6bff86fd1466c7a4257). 2022-08-23 15:46:16,428 - 16912 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskexecutor.TaskExecutor:Un-registering task and sending final execution state FAILED to JobManager for task Source: mysqlsourcefactory (1/1)#0 4ec78812b87fd6bff86fd1466c7a4257. 2022-08-23 15:46:16,440 - 16924 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.Execution:Source: mysqlsourcefactory (1/1) (4ec78812b87fd6bff86fd1466c7a4257) switched from RUNNING to FAILED on b5e95074-82cb-4551-aaf6-3327da83c63b @ 127.0.0.1 (dataPort=-1). com.dtstack.chunjun.throwable.ChunJunRuntimeException: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:111) at com.dtstack.chunjun.source.format.BaseRichInputFormat.openInputFormat(BaseRichInputFormat.java:182) at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:106) ... 5 more 2022-08-23 15:46:16,445 - 16929 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy:Calculating tasks to restart to recover the failed task bc764cd8ddf7a0cff126f51c16239658_0. 2022-08-23 15:46:16,446 - 16930 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy:2 tasks should be restarted to recover the failed task bc764cd8ddf7a0cff126f51c16239658_0. 2022-08-23 15:46:16,450 - 16934 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.ExecutionGraph:Job Flink_Job (235c680366272aa60c6c996eaf429555) switched from state RUNNING to FAILING. org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233) at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224) at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at akka.actor.Actor.aroundReceive(Actor.scala:517) at akka.actor.Actor.aroundReceive$(Actor.scala:515) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) at akka.actor.ActorCell.invoke(ActorCell.scala:561) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) at akka.dispatch.Mailbox.run(Mailbox.scala:225) at akka.dispatch.Mailbox.exec(Mailbox.scala:235) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: com.dtstack.chunjun.throwable.ChunJunRuntimeException: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:111) at com.dtstack.chunjun.source.format.BaseRichInputFormat.openInputFormat(BaseRichInputFormat.java:182) at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:106) ... 5 more 2022-08-23 15:46:16,455 - 16939 INFO [flink-akka.actor.default-dispatcher-6] org.apache.flink.runtime.executiongraph.Execution:Sink: mysqlsinkfactory (1/1) (85c435b87850c0169bdfc868b25718e3) switched from RUNNING to CANCELING. 2022-08-23 15:46:16,456 - 16940 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskmanager.Task:Attempting to cancel task Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3). 2022-08-23 15:46:16,457 - 16941 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskmanager.Task:Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3) switched from RUNNING to CANCELING. 2022-08-23 15:46:16,457 - 16941 INFO [flink-akka.actor.default-dispatcher-5] org.apache.flink.runtime.taskmanager.Task:Triggering cancellation of task code Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3). 2022-08-23 15:46:16,909 - 17393 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.format.BaseRichOutputFormat:initTimingSubmitTask() ,initialDelay:10000, delay:10000, MILLISECONDS Tue Aug 23 15:46:22 CST 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification. 2022-08-23 15:46:22,714 - 23198 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.connector.jdbc.sink.JdbcOutputFormat:write sql:INSERT INTO one_copy1(id, name, age, phone) VALUES (:id, :name, :age, :phone) 2022-08-23 15:46:22,758 - 23242 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.connector.jdbc.sink.JdbcOutputFormat:subTask[0}] wait finished 2022-08-23 15:46:22,830 - 23314 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.format.BaseRichOutputFormat:[MysqlOutputFormat] open successfully, checkpointMode = AT_LEAST_ONCE, checkpointEnabled = false, flushIntervalMills = 10000, batchSize = 1024, [JdbcConf]: { "semantic" : "at-least-once", "errorRecord" : 0, "checkFormat" : true, "parallelism" : 1, "executeDdlAble" : false, "pollingInterval" : 5000, "increment" : false, "flushIntervalMills" : 10000, "polling" : false, "mode" : "insert", "password" : "**", "metricPluginRoot" : "null\metrics", "restoreColumnIndex" : -1, "connection" : [ { "table" : [ "one_copy1" ], "jdbcUrl" : "jdbc:mysql://172.18.8.77:3306/zk_test", "allReplace" : false } ], "table" : "one_copy1", "queryTimeOut" : 0, "fetchSize" : 0, "useMaxFunc" : false, "column" : [ { "name" : "id", "type" : "INT", "index" : 0, "notNull" : false, "part" : false }, { "name" : "name", "type" : "VARCHAR", "index" : 1, "notNull" : false, "part" : false }, { "name" : "age", "type" : "TINYINT", "index" : 2, "notNull" : false, "part" : false }, { "name" : "phone", "type" : "VARCHAR", "index" : 3, "notNull" : false, "part" : false } ], "errorPercentage" : -1, "fieldNameList" : [ ], "withNoLock" : false, "increColumnIndex" : -1, "allReplace" : false, "initReporter" : true, "jdbcUrl" : "jdbc:mysql://172.18.8.77:3306/zk_test", "connectTimeOut" : 0, "batchSize" : 1024, "metricProps" : { "jdbcUrl" : "jdbc:mysql://172.18.8.77:3306/zk_test", "database" : "zk_test", "table" : "test001", "username" : "root", "password" : "**", "properties" : { } }, "speedBytes" : 0, "rowSizeCalculatorType" : "objectSizeCalculator", "metricPluginName" : "mysql", "properties" : { "user" : "root", "password" : "**", "useCursorFetch" : "true", "rewriteBatchedStatements" : "true" }, "username" : "root" } 2022-08-23 15:46:22,831 - 23315 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.format.BaseRichOutputFormat:taskNumber[0] close() 2022-08-23 15:46:42,844 - 43328 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.dirty.log.LogDirtyDataCollector:Print consumer closed. 2022-08-23 15:46:42,844 - 43328 INFO [Sink: mysqlsinkfactory (1/1)#0] com.dtstack.chunjun.sink.format.BaseRichOutputFormat:subtask[0}] close() finished 2022-08-23 15:46:42,845 - 43329 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3) switched from CANCELING to CANCELED. 2022-08-23 15:46:42,845 - 43329 INFO [Sink: mysqlsinkfactory (1/1)#0] org.apache.flink.runtime.taskmanager.Task:Freeing task resources for Sink: mysqlsinkfactory (1/1)#0 (85c435b87850c0169bdfc868b25718e3). 2022-08-23 15:46:42,846 - 43330 INFO [flink-akka.actor.default-dispatcher-7] org.apache.flink.runtime.taskexecutor.TaskExecutor:Un-registering task and sending final execution state CANCELED to JobManager for task Sink: mysqlsinkfactory (1/1)#0 85c435b87850c0169bdfc868b25718e3. 2022-08-23 15:46:42,847 - 43331 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.executiongraph.Execution:Sink: mysqlsinkfactory (1/1) (85c435b87850c0169bdfc868b25718e3) switched from CANCELING to CANCELED. 2022-08-23 15:46:42,849 - 43333 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.executiongraph.ExecutionGraph:Job Flink_Job (235c680366272aa60c6c996eaf429555) switched from state FAILING to FAILED. org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233) at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224) at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at akka.actor.Actor.aroundReceive(Actor.scala:517) at akka.actor.Actor.aroundReceive$(Actor.scala:515) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) at akka.actor.ActorCell.invoke(ActorCell.scala:561) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) at akka.dispatch.Mailbox.run(Mailbox.scala:225) at akka.dispatch.Mailbox.exec(Mailbox.scala:235) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: com.dtstack.chunjun.throwable.ChunJunRuntimeException: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:111) at com.dtstack.chunjun.source.format.BaseRichInputFormat.openInputFormat(BaseRichInputFormat.java:182) at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:106) ... 5 more 2022-08-23 15:46:42,850 - 43334 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.checkpoint.CheckpointCoordinator:Stopping checkpoint coordinator for job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:42,852 - 43336 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.checkpoint.StandaloneCompletedCheckpointStore:Shutting down 2022-08-23 15:46:42,855 - 43339 INFO [flink-akka.actor.default-dispatcher-7] org.apache.flink.runtime.dispatcher.Dispatcher:Job 235c680366272aa60c6c996eaf429555 reached terminal state FAILED. 2022-08-23 15:46:42,855 - 43339 INFO [main] org.apache.flink.runtime.minicluster.MiniCluster:Shutting down Flink Mini Cluster 2022-08-23 15:46:42,855 - 43339 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.TaskExecutor:Stopping TaskExecutor akka://flink/user/rpc/taskmanager_0. 2022-08-23 15:46:42,856 - 43340 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.TaskExecutor:Close ResourceManager connection 5a6e90380996d52815653aba51dced48. 2022-08-23 15:46:42,856 - 43340 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.jobmaster.JobMaster:Stopping the JobMaster for job Flink_Job(235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:42,856 - 43340 INFO [flink-akka.actor.default-dispatcher-11] org.apache.flink.runtime.resourcemanager.ResourceManager:Closing TaskExecutor connection b5e95074-82cb-4551-aaf6-3327da83c63b because: The TaskExecutor is shutting down. 2022-08-23 15:46:42,857 - 43341 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl:Suspending SlotPool. 2022-08-23 15:46:42,857 - 43341 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.TaskExecutor:Close JobManager connection for job 235c680366272aa60c6c996eaf429555. 2022-08-23 15:46:42,857 - 43341 INFO [main] org.apache.flink.runtime.rest.RestServerEndpoint:Shutting down rest endpoint. 2022-08-23 15:46:42,857 - 43341 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.jobmaster.JobMaster:Close ResourceManager connection 5a6e90380996d52815653aba51dced48: Stopping JobMaster for job Flink_Job(235c680366272aa60c6c996eaf429555).. 2022-08-23 15:46:42,857 - 43341 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl:Stopping SlotPool. 2022-08-23 15:46:42,857 - 43341 INFO [flink-akka.actor.default-dispatcher-11] org.apache.flink.runtime.resourcemanager.ResourceManager:Disconnect job manager 9be4ee45fbf0999838ab2685521d4540@akka://flink/user/rpc/jobmanager_3 for job 235c680366272aa60c6c996eaf429555 from the resource manager. 2022-08-23 15:46:42,858 - 43342 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl:Free slot TaskSlot(index:0, state:ALLOCATED, resource profile: ResourceProfile{taskHeapMemory=1024.000gb (1099511627776 bytes), taskOffHeapMemory=1024.000gb (1099511627776 bytes), managedMemory=512.000mb (536870912 bytes), networkMemory=64.000mb (67108864 bytes)}, allocationId: 2df6d256a754041030bc7a2fc03cbcb9, jobId: 235c680366272aa60c6c996eaf429555). 2022-08-23 15:46:42,865 - 43349 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService:Stop job leader service. 2022-08-23 15:46:42,866 - 43350 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager:Shutting down TaskExecutorLocalStateStoresManager. 2022-08-23 15:46:42,868 - 43352 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.io.disk.FileChannelManagerImpl:FileChannelManager removed spill file directory C:\Users\admin\AppData\Local\Temp\flink-io-5658fc3b-33b8-485b-84c4-753623e90653 2022-08-23 15:46:42,868 - 43352 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.io.network.NettyShuffleEnvironment:Shutting down the network environment and its components. 2022-08-23 15:46:42,869 - 43353 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.io.disk.FileChannelManagerImpl:FileChannelManager removed spill file directory C:\Users\admin\AppData\Local\Temp\flink-netty-shuffle-caf0f9b6-89b8-4020-85d7-f97670524c38 2022-08-23 15:46:42,869 - 43353 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.KvStateService:Shutting down the kvState service and its components. 2022-08-23 15:46:42,869 - 43353 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService:Stop job leader service. 2022-08-23 15:46:42,870 - 43354 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.filecache.FileCache:removed file cache directory C:\Users\admin\AppData\Local\Temp\flink-dist-cache-b7cf9ffc-6e03-46c7-b797-97c0ca8e3f84 2022-08-23 15:46:42,871 - 43355 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.taskexecutor.TaskExecutor:Stopped TaskExecutor akka://flink/user/rpc/taskmanager_0. 2022-08-23 15:46:42,889 - 43373 INFO [ForkJoinPool.commonPool-worker-2] org.apache.flink.runtime.webmonitor.WebMonitorEndpoint:Removing cache directory C:\Users\admin\AppData\Local\Temp\flink-web-ui 2022-08-23 15:46:42,892 - 43376 INFO [main] org.apache.flink.runtime.rest.RestServerEndpoint:Shut down complete. 2022-08-23 15:46:42,899 - 43383 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.resourcemanager.ResourceManager:Shut down cluster because application is in CANCELED, diagnostics DispatcherResourceManagerComponent has been closed.. 2022-08-23 15:46:42,901 - 43385 INFO [main] org.apache.flink.runtime.entrypoint.component.DispatcherResourceManagerComponent:Closing components. 2022-08-23 15:46:42,903 - 43387 INFO [main] org.apache.flink.runtime.dispatcher.runner.AbstractDispatcherLeaderProcess:Stopping SessionDispatcherLeaderProcess. 2022-08-23 15:46:42,903 - 43387 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.dispatcher.Dispatcher:Stopping dispatcher akka://flink/user/rpc/dispatcher_2. 2022-08-23 15:46:42,903 - 43387 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.dispatcher.Dispatcher:Stopping all currently running jobs of dispatcher akka://flink/user/rpc/dispatcher_2. 2022-08-23 15:46:42,904 - 43388 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.rest.handler.legacy.backpressure.BackPressureRequestCoordinator:Shutting down back pressure request coordinator. 2022-08-23 15:46:42,904 - 43388 INFO [flink-akka.actor.default-dispatcher-10] org.apache.flink.runtime.dispatcher.Dispatcher:Stopped dispatcher akka://flink/user/rpc/dispatcher_2. 2022-08-23 15:46:42,904 - 43388 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl:Closing the SlotManager. 2022-08-23 15:46:42,904 - 43388 INFO [flink-akka.actor.default-dispatcher-2] org.apache.flink.runtime.resourcemanager.slotmanager.SlotManagerImpl:Suspending the SlotManager. 2022-08-23 15:46:42,907 - 43391 INFO [main] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Stopping Akka RPC service. 2022-08-23 15:46:42,943 - 43427 INFO [flink-metrics-2] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Stopping Akka RPC service. 2022-08-23 15:46:42,943 - 43427 INFO [flink-metrics-2] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Stopped Akka RPC service. 2022-08-23 15:46:42,956 - 43440 INFO [flink-akka.actor.default-dispatcher-11] org.apache.flink.runtime.blob.AbstractBlobCache:Shutting down BLOB cache 2022-08-23 15:46:42,957 - 43441 INFO [flink-akka.actor.default-dispatcher-11] org.apache.flink.runtime.blob.AbstractBlobCache:Shutting down BLOB cache 2022-08-23 15:46:42,959 - 43443 INFO [flink-akka.actor.default-dispatcher-11] org.apache.flink.runtime.blob.BlobServer:Stopped BLOB server at 0.0.0.0:55809 2022-08-23 15:46:42,959 - 43443 INFO [flink-akka.actor.default-dispatcher-11] org.apache.flink.runtime.rpc.akka.AkkaRpcService:Stopped Akka RPC service. Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:144) at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:811) at com.dtstack.chunjun.environment.MyLocalStreamEnvironment.execute(MyLocalStreamEnvironment.java:174) at com.dtstack.chunjun.Main.exeSyncJob(Main.java:227) at com.dtstack.chunjun.Main.main(Main.java:122) at com.dtstack.chunjun.local.test.LocalTest.main(LocalTest.java:174) Caused by: org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118) at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80) at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233) at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224) at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215) at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666) at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89) at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212) at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77) at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26) at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21) at scala.PartialFunction.applyOrElse(PartialFunction.scala:123) at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122) at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172) at akka.actor.Actor.aroundReceive(Actor.scala:517) at akka.actor.Actor.aroundReceive$(Actor.scala:515) at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592) at akka.actor.ActorCell.invoke(ActorCell.scala:561) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258) at akka.dispatch.Mailbox.run(Mailbox.scala:225) at akka.dispatch.Mailbox.exec(Mailbox.scala:235) at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) Caused by: com.dtstack.chunjun.throwable.ChunJunRuntimeException: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:111) at com.dtstack.chunjun.source.format.BaseRichInputFormat.openInputFormat(BaseRichInputFormat.java:182) at com.dtstack.chunjun.source.DtInputFormatSourceFunction.run(DtInputFormatSourceFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:66) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:267) Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.metrics.mysql.MysqlReport at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64) at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74) at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172) at com.dtstack.chunjun.util.DataSyncFactoryUtil.discoverMetric(DataSyncFactoryUtil.java:106) ... 5 more Disconnected from the target VM, address: '127.0.0.1:55801', transport: 'socket'

Process finished with exit code 1

FlechazoW commented 2 years ago

这是想用mysql 作为指标存储嘛？但是为什么 metricPluginConf 的"pluginName" 是 "promethus"呢？