search

datavane / tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI
https://tis.pub
Apache License 2.0
1.04k stars 222 forks source link

整库同步时个别表有json类型字段下游create table ddl报错 #280

Closed charnet1019 closed 9 months ago

charnet1019 commented 1 year ago

mysql-> doris image

11-02 17:29192.168.30.43 17:29:03.874 WARN  c.q.t.p.d.c.BasicDataXRdbmsWriter-TableNotFound{ dbId=doris_mall4cloud_product,tableName='spu'}
11-02 17:29192.168.30.43 17:29:03.874 INFO  c.q.t.p.d.c.BasicDataXRdbmsWriter-create table:spu
   script:CREATE TABLE `spu`
(
    `spu_id`                     BIGINT NOT NULL,
    `create_time`                DATETIMEV2,
    `update_time`                DATETIMEV2,
    `brand_id`                   BIGINT,
    `category_id`                BIGINT,
    `shop_category_id`           BIGINT,
    `shop_id`                    BIGINT,
    `main_img_url`               VARCHAR(765),
    `img_urls`                   VARCHAR(3000),
    `video`                      VARCHAR(450),
    `price_fee`                  BIGINT,
    `market_price_fee`           BIGINT,
    `score_fee`                  BIGINT,
    `status`                     TINYINT,
    `delivery_mode`              VARCHAR(-1073741824),
    `delivery_template_id`       BIGINT,
    `has_sku_img`                TINYINT,
    `spu_type`                   TINYINT,
    `activity_id`                BIGINT,
    `is_compose`                 TINYINT,
    `seq`                        SMALLINT,
    `is_top`                     TINYINT,
    `spu_code`                   VARCHAR(300),
    `supplier_spu_type`          TINYINT,
    `supplier_spu_id`            BIGINT,
    `supplier_delivery_type`     TINYINT,
    `supplier_id`                BIGINT,
    `measure_unit`               VARCHAR(150),
    `sku_choice_status`          TINYINT,
    `is_syn_update`              TINYINT,
    `is_update_msg`              TINYINT,
    `spu_mold`                   TINYINT,
    `is_refund`                  TINYINT,
    `pre_sale_type`              TINYINT,
    `deposit_type`               TINYINT,
    `deposit_amount`             BIGINT,
    `deposit_expansion_amount`   BIGINT,
    `deposit_start_time`         DATETIMEV2,
    `deposit_end_time`           DATETIMEV2,
    `balance_start_time`         DATETIMEV2,
    `balance_end_time`           DATETIMEV2,
    `pre_sale_delivery_type`     TINYINT,
    `pre_sale_delivery_time`     DATETIMEV2,
    `virtual_remark`             VARCHAR(-1073741824),
    `sku_zoned`                  TINYINT,
    `write_off_num`              INT,
    `write_off_multiple_count`   INT,
    `write_off_time`             INT,
    `write_off_start`            DATETIMEV2,
    `write_off_end`              DATETIMEV2
)
 ENGINE=olap
UNIQUE KEY(`spu_id`)
DISTRIBUTED BY HASH(`spu_id`)
BUCKETS 10
PROPERTIES("replication_num" = "1"  )
11-02 17:29192.168.30.43 17:29:03.886 ERROR c.a.datax.core.job.JobContainer-Exception when job run
com.alibaba.datax.common.exception.DataXException: Code:[RdbmsWriter-03], Description:[initialize 'tableName' faild]. - spu - java.lang.RuntimeException: CREATE TABLE `spu`
(
    `spu_id`                     BIGINT NOT NULL,
    `create_time`                DATETIMEV2,
    `update_time`                DATETIMEV2,
    `brand_id`                   BIGINT,
    `category_id`                BIGINT,
    `shop_category_id`           BIGINT,
    `shop_id`                    BIGINT,
    `main_img_url`               VARCHAR(765),
    `img_urls`                   VARCHAR(3000),
    `video`                      VARCHAR(450),
    `price_fee`                  BIGINT,
    `market_price_fee`           BIGINT,
    `score_fee`                  BIGINT,
    `status`                     TINYINT,
    `delivery_mode`              VARCHAR(-1073741824),
    `delivery_template_id`       BIGINT,
    `has_sku_img`                TINYINT,
    `spu_type`                   TINYINT,
    `activity_id`                BIGINT,
    `is_compose`                 TINYINT,
    `seq`                        SMALLINT,
    `is_top`                     TINYINT,
    `spu_code`                   VARCHAR(300),
    `supplier_spu_type`          TINYINT,
    `supplier_spu_id`            BIGINT,
    `supplier_delivery_type`     TINYINT,
    `supplier_id`                BIGINT,
    `measure_unit`               VARCHAR(150),
    `sku_choice_status`          TINYINT,
    `is_syn_update`              TINYINT,
    `is_update_msg`              TINYINT,
    `spu_mold`                   TINYINT,
    `is_refund`                  TINYINT,
    `pre_sale_type`              TINYINT,
    `deposit_type`               TINYINT,
    `deposit_amount`             BIGINT,
    `deposit_expansion_amount`   BIGINT,
    `deposit_start_time`         DATETIMEV2,
    `deposit_end_time`           DATETIMEV2,
    `balance_start_time`         DATETIMEV2,
    `balance_end_time`           DATETIMEV2,
    `pre_sale_delivery_type`     TINYINT,
    `pre_sale_delivery_time`     DATETIMEV2,
    `virtual_remark`             VARCHAR(-1073741824),
    `sku_zoned`                  TINYINT,
    `write_off_num`              INT,
    `write_off_multiple_count`   INT,
    `write_off_time`             INT,
    `write_off_start`            DATETIMEV2,
    `write_off_end`              DATETIMEV2
)
 ENGINE=olap
UNIQUE KEY(`spu_id`)
DISTRIBUTED BY HASH(`spu_id`)
BUCKETS 10
PROPERTIES("replication_num" = "1"  )
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:204)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:150)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.initWriterTable(BasicDataXRdbmsWriter.java:135)
    at com.qlangtech.tis.datax.impl.DataxWriter.process(DataxWriter.java:102)
    at com.alibaba.datax.plugin.writer.doriswriter.TISDorisWriter$Job.init(TISDorisWriter.java:49)
    at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:645)
    at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:299)
    at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:128)
    at com.alibaba.datax.core.Engine.start(Engine.java:92)
    at com.qlangtech.tis.datax.DataxExecutor.startEngine(DataxExecutor.java:461)
    at com.qlangtech.tis.datax.DataxExecutor.entry(DataxExecutor.java:449)
    at com.qlangtech.tis.datax.DataxExecutor.startWork(DataxExecutor.java:352)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:284)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:258)
    at com.qlangtech.tis.datax.DataxExecutor.main(DataxExecutor.java:205)
Caused by: java.sql.SQLException: errCode = 2, detailMessage = Syntax error in line 17:
    `delivery_mode`              VARCHAR(-1073741824),
                                         ^
Encountered: -
Expected: IDENTIFIER

    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2491)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2449)
    at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
    at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:201)
    ... 14 more

    at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33)
    at com.alibaba.datax.plugin.writer.doriswriter.TISDorisWriter$Job.init(TISDorisWriter.java:51)
    at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:645)
    at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:299)
    at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:128)
    at com.alibaba.datax.core.Engine.start(Engine.java:92)
    at com.qlangtech.tis.datax.DataxExecutor.startEngine(DataxExecutor.java:461)
    at com.qlangtech.tis.datax.DataxExecutor.entry(DataxExecutor.java:449)
    at com.qlangtech.tis.datax.DataxExecutor.startWork(DataxExecutor.java:352)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:284)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:258)
    at com.qlangtech.tis.datax.DataxExecutor.main(DataxExecutor.java:205)
Caused by: java.lang.RuntimeException: CREATE TABLE `spu`
(
    `spu_id`                     BIGINT NOT NULL,
    `create_time`                DATETIMEV2,
    `update_time`                DATETIMEV2,
    `brand_id`                   BIGINT,
    `category_id`                BIGINT,
    `shop_category_id`           BIGINT,
    `shop_id`                    BIGINT,
    `main_img_url`               VARCHAR(765),
    `img_urls`                   VARCHAR(3000),
    `video`                      VARCHAR(450),
    `price_fee`                  BIGINT,
    `market_price_fee`           BIGINT,
    `score_fee`                  BIGINT,
    `status`                     TINYINT,
    `delivery_mode`              VARCHAR(-1073741824),
    `delivery_template_id`       BIGINT,
    `has_sku_img`                TINYINT,
    `spu_type`                   TINYINT,
    `activity_id`                BIGINT,
    `is_compose`                 TINYINT,
    `seq`                        SMALLINT,
    `is_top`                     TINYINT,
    `spu_code`                   VARCHAR(300),
    `supplier_spu_type`          TINYINT,
    `supplier_spu_id`            BIGINT,
    `supplier_delivery_type`     TINYINT,
    `supplier_id`                BIGINT,
    `measure_unit`               VARCHAR(150),
    `sku_choice_status`          TINYINT,
    `is_syn_update`              TINYINT,
    `is_update_msg`              TINYINT,
    `spu_mold`                   TINYINT,
    `is_refund`                  TINYINT,
    `pre_sale_type`              TINYINT,
    `deposit_type`               TINYINT,
    `deposit_amount`             BIGINT,
    `deposit_expansion_amount`   BIGINT,
    `deposit_start_time`         DATETIMEV2,
    `deposit_end_time`           DATETIMEV2,
    `balance_start_time`         DATETIMEV2,
    `balance_end_time`           DATETIMEV2,
    `pre_sale_delivery_type`     TINYINT,
    `pre_sale_delivery_time`     DATETIMEV2,
    `virtual_remark`             VARCHAR(-1073741824),
    `sku_zoned`                  TINYINT,
    `write_off_num`              INT,
    `write_off_multiple_count`   INT,
    `write_off_time`             INT,
    `write_off_start`            DATETIMEV2,
    `write_off_end`              DATETIMEV2
)
 ENGINE=olap
UNIQUE KEY(`spu_id`)
DISTRIBUTED BY HASH(`spu_id`)
BUCKETS 10
PROPERTIES("replication_num" = "1"  )
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:204)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:150)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.initWriterTable(BasicDataXRdbmsWriter.java:135)
    at com.qlangtech.tis.datax.impl.DataxWriter.process(DataxWriter.java:102)
    at com.alibaba.datax.plugin.writer.doriswriter.TISDorisWriter$Job.init(TISDorisWriter.java:49)
    ... 10 common frames omitted
Caused by: java.sql.SQLException: errCode = 2, detailMessage = Syntax error in line 17:
    `delivery_mode`              VARCHAR(-1073741824),
                                         ^
Encountered: -
Expected: IDENTIFIER

    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2491)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2449)
    at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
    at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:201)
    ... 14 common frames omitted
11-02 17:29192.168.30.43 17:29:03.891 INFO  c.a.d.c.s.c.c.j.StandAloneJobContainerCommunicator-Total 0 records, 0 bytes | Speed 0B/s, 0 records/s | Error 0 records, 0 bytes |  All Task WaitWriterTime 0.000s |  All Task WaitReaderTime 0.000s | Percentage 0.00%
11-02 17:29192.168.30.43 17:29:03.897 ERROR c.qlangtech.tis.datax.DataxExecutor-com.alibaba.datax.common.exception.DataXException: Code:[RdbmsWriter-03], Description:[initialize 'tableName' faild]. - spu - java.lang.RuntimeException: CREATE TABLE `spu`
(
    `spu_id`                     BIGINT NOT NULL,
    `create_time`                DATETIMEV2,
    `update_time`                DATETIMEV2,
    `brand_id`                   BIGINT,
    `category_id`                BIGINT,
    `shop_category_id`           BIGINT,
    `shop_id`                    BIGINT,
    `main_img_url`               VARCHAR(765),
    `img_urls`                   VARCHAR(3000),
    `video`                      VARCHAR(450),
    `price_fee`                  BIGINT,
    `market_price_fee`           BIGINT,
    `score_fee`                  BIGINT,
    `status`                     TINYINT,
    `delivery_mode`              VARCHAR(-1073741824),
    `delivery_template_id`       BIGINT,
    `has_sku_img`                TINYINT,
    `spu_type`                   TINYINT,
    `activity_id`                BIGINT,
    `is_compose`                 TINYINT,
    `seq`                        SMALLINT,
    `is_top`                     TINYINT,
    `spu_code`                   VARCHAR(300),
    `supplier_spu_type`          TINYINT,
    `supplier_spu_id`            BIGINT,
    `supplier_delivery_type`     TINYINT,
    `supplier_id`                BIGINT,
    `measure_unit`               VARCHAR(150),
    `sku_choice_status`          TINYINT,
    `is_syn_update`              TINYINT,
    `is_update_msg`              TINYINT,
    `spu_mold`                   TINYINT,
    `is_refund`                  TINYINT,
    `pre_sale_type`              TINYINT,
    `deposit_type`               TINYINT,
    `deposit_amount`             BIGINT,
    `deposit_expansion_amount`   BIGINT,
    `deposit_start_time`         DATETIMEV2,
    `deposit_end_time`           DATETIMEV2,
    `balance_start_time`         DATETIMEV2,
    `balance_end_time`           DATETIMEV2,
    `pre_sale_delivery_type`     TINYINT,
    `pre_sale_delivery_time`     DATETIMEV2,
    `virtual_remark`             VARCHAR(-1073741824),
    `sku_zoned`                  TINYINT,
    `write_off_num`              INT,
    `write_off_multiple_count`   INT,
    `write_off_time`             INT,
    `write_off_start`            DATETIMEV2,
    `write_off_end`              DATETIMEV2
)
 ENGINE=olap
UNIQUE KEY(`spu_id`)
DISTRIBUTED BY HASH(`spu_id`)
BUCKETS 10
PROPERTIES("replication_num" = "1"  )
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:204)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:150)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.initWriterTable(BasicDataXRdbmsWriter.java:135)
    at com.qlangtech.tis.datax.impl.DataxWriter.process(DataxWriter.java:102)
    at com.alibaba.datax.plugin.writer.doriswriter.TISDorisWriter$Job.init(TISDorisWriter.java:49)
    at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:645)
    at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:299)
    at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:128)
    at com.alibaba.datax.core.Engine.start(Engine.java:92)
    at com.qlangtech.tis.datax.DataxExecutor.startEngine(DataxExecutor.java:461)
    at com.qlangtech.tis.datax.DataxExecutor.entry(DataxExecutor.java:449)
    at com.qlangtech.tis.datax.DataxExecutor.startWork(DataxExecutor.java:352)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:284)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:258)
    at com.qlangtech.tis.datax.DataxExecutor.main(DataxExecutor.java:205)
Caused by: java.sql.SQLException: errCode = 2, detailMessage = Syntax error in line 17:
    `delivery_mode`              VARCHAR(-1073741824),
                                         ^
Encountered: -
Expected: IDENTIFIER

    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2491)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2449)
    at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
    at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:201)
    ... 14 more

java.lang.Exception: com.alibaba.datax.common.exception.DataXException: Code:[RdbmsWriter-03], Description:[initialize 'tableName' faild]. - spu - java.lang.RuntimeException: CREATE TABLE `spu`
(
    `spu_id`                     BIGINT NOT NULL,
    `create_time`                DATETIMEV2,
    `update_time`                DATETIMEV2,
    `brand_id`                   BIGINT,
    `category_id`                BIGINT,
    `shop_category_id`           BIGINT,
    `shop_id`                    BIGINT,
    `main_img_url`               VARCHAR(765),
    `img_urls`                   VARCHAR(3000),
    `video`                      VARCHAR(450),
    `price_fee`                  BIGINT,
    `market_price_fee`           BIGINT,
    `score_fee`                  BIGINT,
    `status`                     TINYINT,
    `delivery_mode`              VARCHAR(-1073741824),
    `delivery_template_id`       BIGINT,
    `has_sku_img`                TINYINT,
    `spu_type`                   TINYINT,
    `activity_id`                BIGINT,
    `is_compose`                 TINYINT,
    `seq`                        SMALLINT,
    `is_top`                     TINYINT,
    `spu_code`                   VARCHAR(300),
    `supplier_spu_type`          TINYINT,
    `supplier_spu_id`            BIGINT,
    `supplier_delivery_type`     TINYINT,
    `supplier_id`                BIGINT,
    `measure_unit`               VARCHAR(150),
    `sku_choice_status`          TINYINT,
    `is_syn_update`              TINYINT,
    `is_update_msg`              TINYINT,
    `spu_mold`                   TINYINT,
    `is_refund`                  TINYINT,
    `pre_sale_type`              TINYINT,
    `deposit_type`               TINYINT,
    `deposit_amount`             BIGINT,
    `deposit_expansion_amount`   BIGINT,
    `deposit_start_time`         DATETIMEV2,
    `deposit_end_time`           DATETIMEV2,
    `balance_start_time`         DATETIMEV2,
    `balance_end_time`           DATETIMEV2,
    `pre_sale_delivery_type`     TINYINT,
    `pre_sale_delivery_time`     DATETIMEV2,
    `virtual_remark`             VARCHAR(-1073741824),
    `sku_zoned`                  TINYINT,
    `write_off_num`              INT,
    `write_off_multiple_count`   INT,
    `write_off_time`             INT,
    `write_off_start`            DATETIMEV2,
    `write_off_end`              DATETIMEV2
)
 ENGINE=olap
UNIQUE KEY(`spu_id`)
DISTRIBUTED BY HASH(`spu_id`)
BUCKETS 10
PROPERTIES("replication_num" = "1"  )
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:204)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:150)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.initWriterTable(BasicDataXRdbmsWriter.java:135)
    at com.qlangtech.tis.datax.impl.DataxWriter.process(DataxWriter.java:102)
    at com.alibaba.datax.plugin.writer.doriswriter.TISDorisWriter$Job.init(TISDorisWriter.java:49)
    at com.alibaba.datax.core.job.JobContainer.initJobWriter(JobContainer.java:645)
    at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:299)
    at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:128)
    at com.alibaba.datax.core.Engine.start(Engine.java:92)
    at com.qlangtech.tis.datax.DataxExecutor.startEngine(DataxExecutor.java:461)
    at com.qlangtech.tis.datax.DataxExecutor.entry(DataxExecutor.java:449)
    at com.qlangtech.tis.datax.DataxExecutor.startWork(DataxExecutor.java:352)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:284)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:258)
    at com.qlangtech.tis.datax.DataxExecutor.main(DataxExecutor.java:205)
Caused by: java.sql.SQLException: errCode = 2, detailMessage = Syntax error in line 17:
    `delivery_mode`              VARCHAR(-1073741824),
                                         ^
Encountered: -
Expected: IDENTIFIER

    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
    at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
    at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2530)
    at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2683)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2491)
    at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2449)
    at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
    at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
    at com.qlangtech.tis.plugin.datax.common.BasicDataXRdbmsWriter.process(BasicDataXRdbmsWriter.java:201)
    ... 14 more

    at com.qlangtech.tis.datax.DataxExecutor.startWork(DataxExecutor.java:355)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:284)
    at com.qlangtech.tis.datax.DataxExecutor.exec(DataxExecutor.java:258)
    at com.qlangtech.tis.datax.DataxExecutor.main(DataxExecutor.java:205)
Caused by: com.alibaba.datax.common.exception.DataXException: Code:[RdbmsWriter-03], Description:[initialize 'tableName' faild]. - spu - java.lang.RuntimeException: CREATE TABLE `spu`
(
    `spu_id`                     BIGINT NOT NULL,
    `create_time`                DATETIMEV2,
    `update_time`                DATETIMEV2,
    `brand_id`                   BIGINT,
    `category_id`                BIGINT,
    `shop_category_id`           BIGINT,
    `shop_id`                    BIGINT,
    `main_img_url`               VARCHAR(765),
    `img_urls`                   VARCHAR(3000),
    `video`                      VARCHAR(450),
    `price_fee`                  BIGINT,
    `market_price_fee`           BIGINT,
    `score_fee`                  BIGINT,
    `status`                     TINYINT,
    `delivery_mode`              VARCHAR(-1073741824),
    `delivery_template_id`       BIGINT,
    `has_sku_img`                TINYINT,
    `spu_type`                   TINYINT,
    `activity_id`                BIGINT,
    `is_compose`                 TINYINT,
    `seq`                        SMALLINT,
    `is_top`                     TINYINT,
    `spu_code`                   VARCHAR(300),
    `supplier_spu_type`          TINYINT,
    `supplier_spu_id`            BIGINT,
    `supplier_delivery_type`     TINYINT,
    `supplier_id`                BIGINT,
    `measure_unit`               VARCHAR(150),
    `sku_choice_status`          TINYINT,
    `is_syn_update`              TINYINT,
    `is_update_msg`              TINYINT,
    `spu_mold`                   TINYINT,
    `is_refund`                  TINYINT,
    `pre_sale_type`              TINYINT,
    `deposit_type`               TINYINT,
    `deposit_amount`             BIGINT,
    `deposit_expansion_amount`   BIGINT,
    `deposit_start_time`         DATETIMEV2,
    `deposit_end_time`           DATETIMEV2,
    `balance_start_time`         DATETIMEV2,
    `balance_end_time`           DATETIMEV2,
    `pre_sale_delivery_type`     TINYINT,
    `pre_sale_delivery_time`     DATETIMEV2,
    `virtual_remark`             VARCHAR(-1073741824),
    `sku_zoned`                  TINYINT,
    `write_off_num`              INT,
    `write_off_multiple_count`   INT,
    `write_off_time`             INT,
    `write_off_start`            DATETIMEV2,
    `write_off_end`              DATETIMEV2
)
 ENGINE=olap
UNIQUE KEY(`spu_id`)
DISTRIBUTED BY HASH(`spu_id`)
BUCKETS 10
baisui1981 commented 1 year ago

能看查一下 TABLE spu 的delivery_mode 字段在 Source库中是什么类型的不？

charnet1019 commented 1 year ago

能看查一下 TABLE spu 的delivery_mode 字段在 Source库中是什么类型的不？

image 哦 明白了 原来是json类型的 ，这个类型之前没有测试到 ，会修改一下代码。

你可以先手动修改一下TIS自动生成的目标端ddl sql 脚本，先delivery_mode字段的类型先修改一下

baisui1981 commented 9 months ago

已经改成将delivery_mode这样的json类型的字段在下游doris库中映射成 STRING类型