datavane / tis

Support agile DataOps Based on Flink, DataX and Flink-CDC, Chunjun with Web-UI
https://tis.pub
Apache License 2.0
989 stars 217 forks source link

无法识别VARBINARY类型的字段列 #296

Closed zyczw closed 7 months ago

zyczw commented 9 months ago

mysql数据库有一张表里面字段有一个是VARBINARY类型的,现在用tis同步到starrocks里面时报错,发现是tis识别字段类型错误,将VARBINARY识别为varchar类型,然后插入starrokcs失败

表结构:

CREATE TABLE `emp_photo` (
  `id` bigint NOT NULL COMMENT ':',
  `emp_id` bigint DEFAULT NULL COMMENT '人员ID:',
  `photo_sn` tinyint unsigned NOT NULL COMMENT '序号:每人从1开始。',
  `photo_file` varchar(50) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL COMMENT '照片文件名:',
  `photo_url` varchar(255) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci DEFAULT NULL,
--- varbinary
  `photo_data` varbinary(1000) DEFAULT NULL COMMENT '照片数据:',
  `photo_sign` varchar(50) CHARACTER SET utf8mb3 COLLATE utf8mb3_general_ci NOT NULL COMMENT '照片签名数据',
  `is_analyse` tinyint NOT NULL COMMENT '是否已分析',
  `photo_time` datetime NOT NULL COMMENT '照片时间:如果能取到拍摄时间是最好的,如不能就保存添加时间。',
  `is_active` tinyint unsigned NOT NULL COMMENT '是否有效:',
  `quality_score` decimal(6,3) NOT NULL COMMENT '质量分:',
  `create_by` bigint DEFAULT NULL COMMENT '创建人:操作员id',
  `create_time` datetime DEFAULT NULL COMMENT '创建时间',
  `modify_by` bigint DEFAULT NULL COMMENT '修改人:操作员id',
  `modify_time` datetime DEFAULT NULL,
  `use_scene` tinyint DEFAULT NULL,
  `is_deleted` tinyint unsigned DEFAULT NULL COMMENT '是否删除:1删除,0没删',
  `delete_time` datetime DEFAULT NULL COMMENT '删除时间',
  `sort_num` bigint DEFAULT NULL COMMENT '顺序号',
  `time_stamp` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3) COMMENT '更新时间戳',
  PRIMARY KEY (`id`) USING BTREE,
  KEY `ced_emp_photo_ix_empid` (`emp_id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 ROW_FORMAT=DYNAMIC
INSERT INTO `emp_photo` (`id`, `emp_id`, `photo_sn`, `photo_file`, `photo_url`, `photo_data`, `photo_sign`, `is_analyse`, `photo_time`, `is_active`, `quality_score`, `create_by`, `create_time`, `modify_by`, `modify_time`, `use_scene`, `is_deleted`, `delete_time`, `sort_num`, `time_stamp`) VALUES (540219693729120303, 540219698024087560, 1, '', 'photo/0e17bda1-b938-f1807cb07400.jpg', NULL, '44a9a6cd-f67e-43cc-bc35-836d4000c819', 0, '2023-12-05 19:19:47', 0, 0.000, 54021963729120257, '2023-12-05 19:19:47', 540219693729120257, '2023-12-06 10:49:40', 2, 0, NULL, 0, '2023-12-06 10:49:39.960');
baisui1981 commented 7 months ago

目前来看 varbinary 在starrocks 中或者 doris 中没有合适的列类型 ,暂时归到varchar中