search

alibaba / DataX

DataX是阿里云DataWorks数据集成的开源版本。
Other
15.95k stars 5.44k forks source link

sqlserver to databend报错 #1791

Open anna225588 opened 1 year ago

anna225588 commented 1 year ago

[root@mysql-80-24 DataX]# python ./target/datax/datax/bin/datax.py ./sqlserver_xx.json
DataX (DATAX-OPENSOURCE-3.0), From Alibaba ! Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.

2023-05-21 07:38:37.762 [main] INFO MessageSource - JVM TimeZone: GMT+08:00, Locale: zh_CN 2023-05-21 07:38:37.765 [main] INFO MessageSource - use Locale: zh_CN timeZone: sun.util.calendar.ZoneInfo[id="GMT+08:00",offset=28800000,dstSavings=0,useDaylight=false,transitions=0,lastRule=null] 2023-05-21 07:38:37.798 [main] INFO VMInfo - VMInfo# operatingSystem class => sun.management.OperatingSystemImpl 2023-05-21 07:38:37.804 [main] INFO Engine - the machine info => 

    osInfo: Red Hat, Inc. 1.8 25.372-b07
    jvmInfo:        Linux amd64 3.10.0-1160.24.1.el7.x86_64
    cpu num:        16

    totalPhysicalMemory:    -0.00G
    freePhysicalMemory:     -0.00G
    maxFileDescriptorCount: -1
    currentOpenFileDescriptorCount: -1

    GC Names        [PS MarkSweep, PS Scavenge]

    MEMORY_NAME                    | allocation_size                | init_size                      
    PS Eden Space                  | 256.00MB                       | 256.00MB                       
    Code Cache                     | 240.00MB                       | 2.44MB                         
    Compressed Class Space         | 1,024.00MB                     | 0.00MB                         
    PS Survivor Space              | 42.50MB                        | 42.50MB                        
    PS Old Gen                     | 683.00MB                       | 683.00MB                       
    Metaspace                      | -0.00MB                        | 0.00MB

2023-05-21 07:38:37.818 [main] INFO Engine - { "content":[ { "reader":{ "name":"sqlserverreader", "parameter":{ "column":[ "a" ], "connection":[ { "jdbcUrl":[ "jdbc:sqlserver://xxx:xx;DatabaseName=test" ], "table":[ "s" ] } ], "password":"**", "username":"test_user" } }, "writer":{ "name":"databendwriter", "parameter":{ "column":[ "a" ], "connection":[ { "jdbcUrl":"jdbc:databend://xx:8000/migrate_db", "table":"s" } ], "maxBatchRows":65536, "maxBatchSize":134217728, "password":"*****", "postSql":[

                                    ],
                                    "preSql":[

                                    ],
                                    "username":"xx"
                            }
                    }
            }
    ],
    "setting":{
            "speed":{
                    "channel":""
            }
    }

}

2023-05-21 07:38:37.838 [main] INFO PerfTrace - PerfTrace traceId=job_-1, isEnable=false 2023-05-21 07:38:37.839 [main] INFO JobContainer - DataX jobContainer starts job. 2023-05-21 07:38:37.840 [main] INFO JobContainer - Set jobId = 0 2023-05-21 07:38:38.368 [job-0] INFO OriginalConfPretreatmentUtil - Available jdbcUrl:jdbc:sqlserver://172.22.22.67:52016;DatabaseName=test. 2023-05-21 07:38:38.463 [job-0] INFO OriginalConfPretreatmentUtil - table:[s] has columns:[a]. 2023-05-21 07:38:38.484 [job-0] ERROR JobContainer - Exception when job run java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:435) ~[datax-common-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:70) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:34) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:41) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.writer.databendwriter.DatabendWriter$Job.init(DatabendWriter.java:36) ~[databendwriter-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704) ~[datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304) ~[datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) ~[datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.start(Engine.java:86) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.entry(Engine.java:168) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.main(Engine.java:201) [datax-core-0.0.1-SNAPSHOT.jar:na] 2023-05-21 07:38:38.489 [job-0] INFO StandAloneJobContainerCommunicator - Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes | All Task WaitWriterTime 0.000s | All Task WaitReaderTime 0.000s | Percentage 0.00% 2023-05-21 07:38:38.495 [job-0] ERROR Engine -

经DataX智能分析,该任务最可能的错误原因是: com.alibaba.datax.common.exception.DataXException: Code:[Framework-02], Description:[DataX引擎运行过程出错，具体原因请参看DataX运行结束时的错误诊断信息 .]. - java.lang.ClassCastException: java.lang.String cannot be cast to java.util.List at com.alibaba.datax.common.util.Configuration.getList(Configuration.java:435) at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:70) at com.alibaba.datax.plugin.rdbms.writer.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:34) at com.alibaba.datax.plugin.rdbms.writer.CommonRdbmsWriter$Job.init(CommonRdbmsWriter.java:41) at com.alibaba.datax.plugin.writer.databendwriter.DatabendWriter$Job.init(DatabendWriter.java:36) at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:704) at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:304) at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) at com.alibaba.datax.core.Engine.start(Engine.java:86) at com.alibaba.datax.core.Engine.entry(Engine.java:168) at com.alibaba.datax.core.Engine.main(Engine.java:201)

FuYouJ commented 1 year ago

你这个在解析配置的时候就错了检查洗配置吧