Open nglszhaosi opened 2 months ago
![Uploading image.png…]() 下面是日志，不知道问题出在哪里 2024-05-20 20:02:45 [BuildCommand.buildDataXParam-100] ------------------Command parameters: 2024-05-20 20:02:45 [ExecutorJobHandler.execute-57] ------------------DataX process id: 78670 2024-05-20 20:02:45 [ProcessCallbackThread.callbackLog-186] ----------- datax-web job callback finish. 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] DataX (DATAX-OPENSOURCE-3.0), From Alibaba ! 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved. 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.421 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.427 [main] INFO Engine - the machine info => 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] osInfo: Red Hat, Inc. 1.8 25.402-b06 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] jvmInfo: Linux amd64 3.10.0-1160.114.2.el7.x86_64 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] cpu num: 16 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] totalPhysicalMemory: -0.00G 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] freePhysicalMemory: -0.00G 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] maxFileDescriptorCount: -1 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] currentOpenFileDescriptorCount: -1 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] GC Names [PS MarkSweep, PS Scavenge] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] MEMORY_NAME | allocation_size | init_size 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] PS Eden Space | 256.00MB | 256.00MB 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] Code Cache | 240.00MB | 2.44MB 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] Compressed Class Space | 1,024.00MB | 0.00MB 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] PS Survivor Space | 42.50MB | 42.50MB 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] PS Old Gen | 683.00MB | 683.00MB 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] Metaspace | -0.00MB | 0.00MB 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.444 [main] INFO Engine - 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "content":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "reader":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"mysqlreader", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "parameter":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "column":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "id", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "disease_name", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "disease_pic_path" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "connection":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "jdbcUrl":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "jdbc:mysql://192.168.16.173:3306/bs" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "table":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "wheat_disease" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "password":"****", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "splitPk":"", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "username":"root" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "writer":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"hbase11xwriter", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "parameter":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "column":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "index":1, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"0:disease_name", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "type":"string" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "index":2, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"0:disease_pic_path", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "type":"string" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "hbaseConfig":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "hbase.zookeeper.quorum":"hadoop01:2181" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "mode":"normal", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "rowkeyColumn":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "index":0, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "type":"string" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "table":"BS:WHEATDISEASE" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "setting":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "errorLimit":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "percentage":0.02, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "record":0 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "speed":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "byte":1048576, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "channel":3 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.462 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.464 [main] INFO PerfTrace - PerfTrace traceId=job-1, isEnable=false, priority=0 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.464 [main] INFO JobContainer - DataX jobContainer starts job. 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.465 [main] INFO JobContainer - Set jobId = 0 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.253 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.16.173:3306/bs?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.521 [job-0] INFO OriginalConfPretreatmentUtil - table:[wheat_disease] has columns:[id,disease_name,disease_pic_path]. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.532 [job-0] INFO JobContainer - jobContainer starts to do prepare ... 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.533 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work . 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.533 [job-0] INFO JobContainer - DataX Writer.Job [hbase11xwriter] do prepare work . 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.533 [job-0] INFO JobContainer - jobContainer starts to do split ... 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.534 [job-0] INFO JobContainer - Job set Max-Byte-Speed to 1048576 bytes. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.538 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [1] tasks. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.538 [job-0] INFO JobContainer - DataX Writer.Job [hbase11xwriter] splits to [1] tasks. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.549 [job-0] INFO JobContainer - jobContainer starts to do schedule ... 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.551 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.553 [job-0] INFO JobContainer - Running by standalone Mode. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.559 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.566 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.566 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.577 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.580 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [select id,disease_name,disease_pic_path from wheat_disease 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] ] jdbcUrl:[jdbc:mysql://192.168.16.173:3306/bs?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.616 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [select id,disease_name,disease_pic_path from wheat_disease 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] ] jdbcUrl:[jdbc:mysql://192.168.16.173:3306/bs?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]. 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.117 [0-0-0-writer] INFO ZooKeeper - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:host.name=hadoop01 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.version=1.8.0_402 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.vendor=Red Hat, Inc. 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.class.path=/export/servers/datax/lib/._commons-logging-1.1.1.jar:/export/servers/datax/lib/commons-logging-1.1.1.jar:/export/servers/datax/lib/._httpcore-4.4.jar:/export/servers/datax/lib/httpcore-4.4.jar:/export/servers/datax/lib/._fastjson-1.1.46.sec01.jar:/export/servers/datax/lib/fastjson-1.1.46.sec01.jar:/export/servers/datax/lib/._commons-configuration-1.10.jar:/export/servers/datax/lib/commons-configuration-1.10.jar:/export/servers/datax/lib/._httpclient-4.4.jar:/export/servers/datax/lib/httpclient-4.4.jar:/export/servers/datax/lib/._commons-cli-1.2.jar:/export/servers/datax/lib/commons-cli-1.2.jar:/export/servers/datax/lib/._fluent-hc-4.4.jar:/export/servers/datax/lib/fluent-hc-4.4.jar:/export/servers/datax/lib/._commons-lang3-3.3.2.jar:/export/servers/datax/lib/commons-lang3-3.3.2.jar:/export/servers/datax/lib/._janino-2.5.16.jar:/export/servers/datax/lib/janino-2.5.16.jar:/export/servers/datax/lib/._commons-codec-1.9.jar:/export/servers/datax/lib/commons-codec-1.9.jar:/export/servers/datax/lib/._logback-core-1.0.13.jar:/export/servers/datax/lib/logback-core-1.0.13.jar:/export/servers/datax/lib/._groovy-all-2.1.9.jar:/export/servers/datax/lib/groovy-all-2.1.9.jar:/export/servers/datax/lib/._commons-io-2.4.jar:/export/servers/datax/lib/commons-io-2.4.jar:/export/servers/datax/lib/._datax-common-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/datax-common-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/._hamcrest-core-1.3.jar:/export/servers/datax/lib/hamcrest-core-1.3.jar:/export/servers/datax/lib/._commons-collections-3.2.1.jar:/export/servers/datax/lib/commons-collections-3.2.1.jar:/export/servers/datax/lib/._datax-transformer-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/datax-transformer-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/._commons-lang-2.6.jar:/export/servers/datax/lib/commons-lang-2.6.jar:/export/servers/datax/lib/._commons-beanutils-1.9.2.jar:/export/servers/datax/lib/commons-beanutils-1.9.2.jar:/export/servers/datax/lib/._logback-classic-1.0.13.jar:/export/servers/datax/lib/logback-classic-1.0.13.jar:/export/servers/datax/lib/._commons-math3-3.1.1.jar:/export/servers/datax/lib/commons-math3-3.1.1.jar:/export/servers/datax/lib/._datax-core-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/datax-core-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/._slf4j-api-1.7.10.jar:/export/servers/datax/lib/slf4j-api-1.7.10.jar:. 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:java.io.tmpdir=/tmp 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:java.compiler= 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:os.name=Linux 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:os.arch=amd64 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:os.version=3.10.0-1160.114.2.el7.x86_64 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:user.name=root 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:user.home=/root 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:user.dir=/export/servers/datax-web-2.1.2/bin 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.120 [0-0-0-writer] INFO ZooKeeper - Initiating client connection, connectString=hadoop01:2181 sessionTimeout=90000 watcher=hconnection-0x68497ed90x0, quorum=hadoop01:2181, baseZNode=/hbase 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.139 [0-0-0-writer-SendThread(hadoop01:2181)] INFO ClientCnxn - Opening socket connection to server hadoop01/192.168.16.130:2181. Will not attempt to authenticate using SASL (unknown error) 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.143 [0-0-0-writer-SendThread(hadoop01:2181)] INFO ClientCnxn - Socket connection established to hadoop01/192.168.16.130:2181, initiating session 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.151 [0-0-0-writer-SendThread(hadoop01:2181)] INFO ClientCnxn - Session establishment complete on server hadoop01/192.168.16.130:2181, sessionid = 0xff8f89d7afb6003e, negotiated timeout = 40000 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.682 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[1108]ms 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.682 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.575 [job-0] INFO StandAloneJobContainerCommunicator - Total 8 records, 700 bytes | Speed 70B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00% 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO JobContainer - DataX Writer.Job [hbase11xwriter] do post work. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO JobContainer - DataX jobId [0] completed successfully. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.577 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /export/servers/datax/hook 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.577 [job-0] INFO JobContainer - 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] [total cpu info] => 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] averageCpu | maxDeltaCpu | minDeltaCpu 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] -1.00% | -1.00% | -1.00% 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] [total gc info] => 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] PS MarkSweep | 1 | 1 | 1 | 0.162s | 0.162s | 0.162s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] PS Scavenge | 1 | 1 | 1 | 0.060s | 0.060s | 0.060s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.577 [job-0] INFO JobContainer - PerfTrace not enable! 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.578 [job-0] INFO StandAloneJobContainerCommunicator - Total 8 records, 700 bytes | Speed 70B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00% 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.579 [job-0] INFO JobContainer - 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务启动时刻 : 2024-05-20 20:02:46 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务结束时刻 : 2024-05-20 20:02:57 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务总计耗时 : 11s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务平均流量 : 70B/s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 记录写入速度 : 0rec/s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 读出记录总数 : 8 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 读写失败总数 : 0 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] Loading class com.mysql.jdbc.Driver'. This is deprecated. The new driver class iscom.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] May 20, 2024 8:02:47 PM org.apache.hadoop.util.NativeCodeLoader 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] May 20, 2024 8:02:48 PM org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] INFO: Process identifier=hconnection-0x68497ed9 connecting to ZooKeeper ensemble=hadoop01:2181 2024-05-20 20:02:57 [JobThread.run-165] ----------- datax-web job execute end(finish) ---------------------- ReturnT:ReturnT [code=200, msg=LogStatistics{taskStartTime=2024-05-20 20:02:46, taskEndTime=2024-05-20 20:02:57, taskTotalTime=11s, taskAverageFlow=70B/s, taskRecordWritingSpeed=0rec/s, taskRecordReaderNum=8, taskRecordWriteFailNum=0}, content=null] 2024-05-20 20:02:58 [TriggerCallbackThread.callbackLog-186] ----------- datax-web job callback finish.
id
disease_name
disease_pic_path
com.mysql.jdbc.Driver'. This is deprecated. The new driver class is
![Uploading image.png…]() 下面是日志，不知道问题出在哪里 2024-05-20 20:02:45 [BuildCommand.buildDataXParam-100] ------------------Command parameters: 2024-05-20 20:02:45 [ExecutorJobHandler.execute-57] ------------------DataX process id: 78670 2024-05-20 20:02:45 [ProcessCallbackThread.callbackLog-186]
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:os.name=Linux
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:os.arch=amd64
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:os.version=3.10.0-1160.114.2.el7.x86_64
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:user.name=root
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:user.home=/root
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:user.dir=/export/servers/datax-web-2.1.2/bin
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.120 [0-0-0-writer] INFO ZooKeeper - Initiating client connection, connectString=hadoop01:2181 sessionTimeout=90000 watcher=hconnection-0x68497ed90x0, quorum=hadoop01:2181, baseZNode=/hbase
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.139 [0-0-0-writer-SendThread(hadoop01:2181)] INFO ClientCnxn - Opening socket connection to server hadoop01/192.168.16.130:2181. Will not attempt to authenticate using SASL (unknown error)
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.143 [0-0-0-writer-SendThread(hadoop01:2181)] INFO ClientCnxn - Socket connection established to hadoop01/192.168.16.130:2181, initiating session
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.151 [0-0-0-writer-SendThread(hadoop01:2181)] INFO ClientCnxn - Session establishment complete on server hadoop01/192.168.16.130:2181, sessionid = 0xff8f89d7afb6003e, negotiated timeout = 40000
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.682 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] is successed, used[1108]ms
2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.682 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] completed it's tasks.
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.575 [job-0] INFO StandAloneJobContainerCommunicator - Total 8 records, 700 bytes | Speed 70B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00%
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO AbstractScheduler - Scheduler accomplished all tasks.
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO JobContainer - DataX Writer.Job [hbase11xwriter] do post work.
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do post work.
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.576 [job-0] INFO JobContainer - DataX jobId [0] completed successfully.
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.577 [job-0] INFO HookInvoker - No hook invoked, because base dir not exists or is a file: /export/servers/datax/hook
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.577 [job-0] INFO JobContainer -
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] [total cpu info] =>
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] averageCpu | maxDeltaCpu | minDeltaCpu
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] WARNING: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] May 20, 2024 8:02:48 PM org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] INFO: Process identifier=hconnection-0x68497ed9 connecting to ZooKeeper ensemble=hadoop01:2181
2024-05-20 20:02:57 [JobThread.run-165]
----------- datax-web job callback finish. 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] DataX (DATAX-OPENSOURCE-3.0), From Alibaba ! 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved. 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:45 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.421 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.427 [main] INFO Engine - the machine info => 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] osInfo: Red Hat, Inc. 1.8 25.402-b06 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] jvmInfo: Linux amd64 3.10.0-1160.114.2.el7.x86_64 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] cpu num: 16 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] totalPhysicalMemory: -0.00G 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] freePhysicalMemory: -0.00G 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] maxFileDescriptorCount: -1 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] currentOpenFileDescriptorCount: -1 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] GC Names [PS MarkSweep, PS Scavenge] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] MEMORY_NAME | allocation_size | init_size
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] PS Eden Space | 256.00MB | 256.00MB
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] Code Cache | 240.00MB | 2.44MB
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] Compressed Class Space | 1,024.00MB | 0.00MB
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] PS Survivor Space | 42.50MB | 42.50MB
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] PS Old Gen | 683.00MB | 683.00MB
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] Metaspace | -0.00MB | 0.00MB
2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.444 [main] INFO Engine - 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "content":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "reader":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"mysqlreader", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "parameter":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "column":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "
id", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "disease_name", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "disease_pic_path" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "connection":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "jdbcUrl":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "jdbc:mysql://192.168.16.173:3306/bs" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "table":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "wheat_disease" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "password":"****", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "splitPk":"", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "username":"root" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "writer":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"hbase11xwriter", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "parameter":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "column":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "index":1, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"0:disease_name", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "type":"string" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "index":2, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "name":"0:disease_pic_path", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "type":"string" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "hbaseConfig":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "hbase.zookeeper.quorum":"hadoop01:2181" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "mode":"normal", 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "rowkeyColumn":[ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] { 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "index":0, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "type":"string" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "table":"BS:WHEATDISEASE" 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] ], 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "setting":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "errorLimit":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "percentage":0.02, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "record":0 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] }, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "speed":{ 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "byte":1048576, 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] "channel":3 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] } 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.462 [main] WARN Engine - prioriy set to 0, because NumberFormatException, the value is: null 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.464 [main] INFO PerfTrace - PerfTrace traceId=job-1, isEnable=false, priority=0 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.464 [main] INFO JobContainer - DataX jobContainer starts job. 2024-05-20 20:02:46 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:46.465 [main] INFO JobContainer - Set jobId = 0 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.253 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:mysql://192.168.16.173:3306/bs?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.521 [job-0] INFO OriginalConfPretreatmentUtil - table:[wheat_disease] has columns:[id,disease_name,disease_pic_path]. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.532 [job-0] INFO JobContainer - jobContainer starts to do prepare ... 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.533 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] do prepare work . 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.533 [job-0] INFO JobContainer - DataX Writer.Job [hbase11xwriter] do prepare work . 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.533 [job-0] INFO JobContainer - jobContainer starts to do split ... 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.534 [job-0] INFO JobContainer - Job set Max-Byte-Speed to 1048576 bytes. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.538 [job-0] INFO JobContainer - DataX Reader.Job [mysqlreader] splits to [1] tasks. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.538 [job-0] INFO JobContainer - DataX Writer.Job [hbase11xwriter] splits to [1] tasks. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.549 [job-0] INFO JobContainer - jobContainer starts to do schedule ... 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.551 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.553 [job-0] INFO JobContainer - Running by standalone Mode. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.559 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [1] channels for [1] tasks. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.566 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.566 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.577 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[0] attemptCount[1] is started 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.580 [0-0-0-reader] INFO CommonRdbmsReader$Task - Begin to read record by Sql: [selectid,disease_name,disease_pic_pathfrom wheat_disease 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] ] jdbcUrl:[jdbc:mysql://192.168.16.173:3306/bs?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]. 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:47.616 [0-0-0-reader] INFO CommonRdbmsReader$Task - Finished read record by Sql: [selectid,disease_name,disease_pic_pathfrom wheat_disease 2024-05-20 20:02:47 [AnalysisStatistics.analysisStatisticsLog-53] ] jdbcUrl:[jdbc:mysql://192.168.16.173:3306/bs?yearIsDateType=false&zeroDateTimeBehavior=convertToNull&tinyInt1isBit=false&rewriteBatchedStatements=true]. 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.117 [0-0-0-writer] INFO ZooKeeper - Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:host.name=hadoop01 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.version=1.8.0_402 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.vendor=Red Hat, Inc. 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.402.b06-1.el7_9.x86_64/jre 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.118 [0-0-0-writer] INFO ZooKeeper - Client environment:java.class.path=/export/servers/datax/lib/._commons-logging-1.1.1.jar:/export/servers/datax/lib/commons-logging-1.1.1.jar:/export/servers/datax/lib/._httpcore-4.4.jar:/export/servers/datax/lib/httpcore-4.4.jar:/export/servers/datax/lib/._fastjson-1.1.46.sec01.jar:/export/servers/datax/lib/fastjson-1.1.46.sec01.jar:/export/servers/datax/lib/._commons-configuration-1.10.jar:/export/servers/datax/lib/commons-configuration-1.10.jar:/export/servers/datax/lib/._httpclient-4.4.jar:/export/servers/datax/lib/httpclient-4.4.jar:/export/servers/datax/lib/._commons-cli-1.2.jar:/export/servers/datax/lib/commons-cli-1.2.jar:/export/servers/datax/lib/._fluent-hc-4.4.jar:/export/servers/datax/lib/fluent-hc-4.4.jar:/export/servers/datax/lib/._commons-lang3-3.3.2.jar:/export/servers/datax/lib/commons-lang3-3.3.2.jar:/export/servers/datax/lib/._janino-2.5.16.jar:/export/servers/datax/lib/janino-2.5.16.jar:/export/servers/datax/lib/._commons-codec-1.9.jar:/export/servers/datax/lib/commons-codec-1.9.jar:/export/servers/datax/lib/._logback-core-1.0.13.jar:/export/servers/datax/lib/logback-core-1.0.13.jar:/export/servers/datax/lib/._groovy-all-2.1.9.jar:/export/servers/datax/lib/groovy-all-2.1.9.jar:/export/servers/datax/lib/._commons-io-2.4.jar:/export/servers/datax/lib/commons-io-2.4.jar:/export/servers/datax/lib/._datax-common-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/datax-common-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/._hamcrest-core-1.3.jar:/export/servers/datax/lib/hamcrest-core-1.3.jar:/export/servers/datax/lib/._commons-collections-3.2.1.jar:/export/servers/datax/lib/commons-collections-3.2.1.jar:/export/servers/datax/lib/._datax-transformer-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/datax-transformer-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/._commons-lang-2.6.jar:/export/servers/datax/lib/commons-lang-2.6.jar:/export/servers/datax/lib/._commons-beanutils-1.9.2.jar:/export/servers/datax/lib/commons-beanutils-1.9.2.jar:/export/servers/datax/lib/._logback-classic-1.0.13.jar:/export/servers/datax/lib/logback-classic-1.0.13.jar:/export/servers/datax/lib/._commons-math3-3.1.1.jar:/export/servers/datax/lib/commons-math3-3.1.1.jar:/export/servers/datax/lib/._datax-core-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/datax-core-0.0.1-SNAPSHOT.jar:/export/servers/datax/lib/._slf4j-api-1.7.10.jar:/export/servers/datax/lib/slf4j-api-1.7.10.jar:. 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:java.io.tmpdir=/tmp 2024-05-20 20:02:48 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:48.119 [0-0-0-writer] INFO ZooKeeper - Client environment:java.compiler=2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] -1.00% | -1.00% | -1.00% 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53]
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] [total gc info] => 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] NAME | totalGCCount | maxDeltaGCCount | minDeltaGCCount | totalGCTime | maxDeltaGCTime | minDeltaGCTime
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] PS MarkSweep | 1 | 1 | 1 | 0.162s | 0.162s | 0.162s
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] PS Scavenge | 1 | 1 | 1 | 0.060s | 0.060s | 0.060s
2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.577 [job-0] INFO JobContainer - PerfTrace not enable! 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.578 [job-0] INFO StandAloneJobContainerCommunicator - Total 8 records, 700 bytes | Speed 70B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 100.00% 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57.579 [job-0] INFO JobContainer - 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务启动时刻 : 2024-05-20 20:02:46 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务结束时刻 : 2024-05-20 20:02:57 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务总计耗时 : 11s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 任务平均流量 : 70B/s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 记录写入速度 : 0rec/s 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 读出记录总数 : 8 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 读写失败总数 : 0 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] Loading class
com.mysql.jdbc.Driver'. This is deprecated. The new driver class iscom.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary. 2024-05-20 20:02:57 [AnalysisStatistics.analysisStatisticsLog-53] May 20, 2024 8:02:47 PM org.apache.hadoop.util.NativeCodeLoader----------- datax-web job execute end(finish) -----------
----------- ReturnT:ReturnT [code=200, msg=LogStatistics{taskStartTime=2024-05-20 20:02:46, taskEndTime=2024-05-20 20:02:57, taskTotalTime=11s, taskAverageFlow=70B/s, taskRecordWritingSpeed=0rec/s, taskRecordReaderNum=8, taskRecordWriteFailNum=0}, content=null] 2024-05-20 20:02:58 [TriggerCallbackThread.callbackLog-186]
----------- datax-web job callback finish.