Altinity / clickhouse-mysql-data-reader

utility to read mysql data
MIT License
369 stars 94 forks source link

when I insert a row in mysql ,I can not search it on clickhouse #145

Closed justlooks closed 4 years ago

justlooks commented 5 years ago

the clickhouse catch the insert event ,but failed to insert into clickhouse

2019-01-18 18:01:45,726/1547805705.726506:DEBUG:starting tail -n +2 'repl_d_all_b_1547805705.7249274_70f83868-9d29-46d8-8f21-8338d1fb91d0.csv' | clickhouse-client  --host=127.0.0.1 --port=9000 --user=default --query='INSERT INTO `test`.`legend_order_info` (`buy_time`, `car_alias`, `car_brand`, `car_brand_id`, `car_color`, `car_company`, `car_license`, `car_models`, `car_models_id`, `car_power`, `car_power_id`, `car_series`, `car_series_id`, `car_year`, `car_year_id`, `channel_id`, `channel_name`, `company`, `confirm_time`, `contact_mobile`, `contact_name`, `create_time`, `creator`, `customer_address`, `customer_car_id`, `customer_id`, `customer_mobile`, `customer_name`, `discount`, `down_payment`, `engine_no`, `expect_time`, `expected_time`, `fee_amount`, `fee_discount`, `finish_time`, `gmt_create`, `gmt_modified`, `goods_amount`, `goods_count`, `goods_discount`, `id`, `img_url`, `import_info`, `insurance_company_id`, `insurance_company_name`, `invoice_type`, `is_deleted`, `is_lock`, `is_notice`, `is_visit`, `mileage`, `modifier`, `oil_meter`, `operator_name`, `order_amount`, `order_discount_amount`, `order_sn`, `order_status`, `order_tag`, `order_type`, `other_insurance_company_id`, `other_insurance_company_name`, `parent_id`, `pay_amount`, `pay_status`, `pay_time`, `postscript`, `pre_coupon_amount`, `pre_discount_rate`, `pre_preferentia_amount`, `pre_tax_amount`, `pre_total_amount`, `proxy_type`, `receiver`, `receiver_name`, `refer`, `service_amount`, `service_count`, `service_discount`, `shop_id`, `sign_amount`, `tax_amount`, `total_amount`, `upkeep_mileage`, `ver`, `vin`) FORMAT CSV'
Code: 27. DB::Exception: Cannot parse input: expected , before: foxhch,,,,2019-01-20,0,,0,0,,,,10.13,,,,0.00,0.00,,,,0.00,0,0.00,123945,,,0,,0,N,N,0,0,,0,,,0.00,0.00,,haha,20,0,0,,0,0.00,0,,,0.00,1.00000000,0.00,0.00,0.00,14: (at row 1)

Row 1:
Column 0,   name: buy_time,                     type: Nullable(DateTime),       parsed text: ",,,0,,,,,0,,0,,0,,0"
Column 1,   name: car_alias,                    type: Nullable(String),         parsed text: "13"
Column 2,   name: car_brand,                    type: Nullable(String),         parsed text: "mychannel"
Column 3,   name: car_brand_id,                 type: Nullable(Int32),          parsed text: <EMPTY>
ERROR: garbage after Nullable(Int32): "foxhch,,,,"

# in mysql

root@localhost:test 06:01:21>insert into legend_order_info (id,shop_id,order_status,order_tag,refer,create_time,proxy_type,channel_id,channel_name,down_payment,company) values(123945,123239,'haha',20,11,'2019-01-20',14,13,'mychannel',10.13,'foxhch');
Query OK, 1 row affected (0.01 sec)

# the ddl on clickhouse

t-k8s-a1 :) show create table legend_order_info\G

SHOW CREATE TABLE legend_order_info

Row 1:
──────
statement: CREATE TABLE test.legend_order_info ( id Int32,  creator Nullable(Int32),  modifier Nullable(Int32),  gmt_create Nullable(DateTime),  pay_time Nullable(DateTime),  gmt_modified Nullable(DateTime),  is_deleted Nullable(String),  shop_id Int32,  order_type Nullable(Int32),  parent_id Nullable(Int32),  order_sn Nullable(String),  customer_id Nullable(Int32),  customer_car_id Nullable(Int32),  expected_time Nullable(DateTime),  goods_amount Nullable(Decimal(12, 2)),  service_amount Nullable(Decimal(12, 2)),  tax_amount Nullable(Decimal(12, 2)),  total_amount Nullable(Decimal(12, 2)),  postscript Nullable(String),  order_status String,  car_license Nullable(String),  car_brand_id Nullable(Int32),  car_series_id Nullable(Int32),  car_power_id Nullable(Int32),  car_year_id Nullable(Int32),  car_models_id Nullable(Int32),  car_brand Nullable(String),  car_series Nullable(String),  car_power Nullable(String),  car_year Nullable(String),  car_models Nullable(String),  car_company Nullable(String),  import_info Nullable(String),  customer_name Nullable(String),  customer_mobile Nullable(String),  vin Nullable(String),  engine_no Nullable(String),  receiver Nullable(Int32),  invoice_type Nullable(Int32),  discount Nullable(Decimal(12, 2)),  receiver_name Nullable(String),  operator_name Nullable(String),  order_amount Nullable(Decimal(12, 2)),  pay_status Nullable(UInt8),  finish_time Nullable(DateTime),  goods_count Nullable(Int32),  service_count Nullable(Int32),  car_alias Nullable(String),  pre_discount_rate Nullable(Decimal(12, 2)),  pre_tax_amount Nullable(Decimal(12, 2)),  pre_preferentia_amount Nullable(Decimal(12, 2)),  pre_coupon_amount Nullable(Decimal(12, 2)),  pre_total_amount Nullable(Decimal(12, 2)),  pay_amount Nullable(Decimal(12, 2)),  sign_amount Nullable(Decimal(12, 2)),  other_insurance_company_id Nullable(Int32),  other_insurance_company_name Nullable(String),  goods_discount Nullable(Decimal(12, 2)),  service_discount Nullable(Decimal(12, 2)),  fee_amount Nullable(Decimal(12, 2)),  fee_discount Nullable(Decimal(12, 2)),  contact_name Nullable(String),  contact_mobile Nullable(String),  insurance_company_id Nullable(Int32),  insurance_company_name Nullable(String),  mileage Nullable(String),  car_color Nullable(String),  buy_time Nullable(DateTime),  customer_address Nullable(String),  oil_meter Nullable(String),  is_notice Nullable(Int8),  is_visit Nullable(Int32),  is_lock Nullable(String),  img_url Nullable(String),  order_tag UInt8,  ver Nullable(String),  refer UInt8,  upkeep_mileage Nullable(String),  create_time Date,  order_discount_amount Nullable(Decimal(16, 2)),  confirm_time Nullable(DateTime),  proxy_type UInt8,  channel_id UInt32,  channel_name String,  expect_time Nullable(DateTime),  down_payment Decimal(12, 2),  company Nullable(String)) ENGINE = MergeTree(create_time, (id, create_time), 8192)

1 rows in set. Elapsed: 0.002 sec.