search

pingcap / tidb

TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try AI-powered Chat2Query free at : https://www.pingcap.com/tidb-serverless/
https://pingcap.com
Apache License 2.0
37.13k stars 5.83k forks source link

WriteConflict when using Sqoop to export data from hdfs to tidb #6434

Closed Lyle-liang closed 6 years ago

Lyle-liang commented 6 years ago

Please answer the question, thanks!

  1. sqoop export --connect 'jdbc:mysql://host:4000/db' --username 'root' --table testTable --export-dir 'hdfs://test/' --input-fields-terminated-by '\001' --lines-terminated-by '\n' --input-null-string '\N' --input-null-non-string '\N' --update-mode allowinsert --update-key id -m 3

  2. error log image

  3. How to deal with it？

zhexuany commented 6 years ago

@Lyle-liang It seems like you tried to insert same data twice? If that is the case, please try drop table and retry insert again.

Lyle-liang commented 6 years ago

@zhexuany the table is synchronizing data in real time，so what you said can not do. Do you have another solution？

coocood commented 6 years ago

@Lyle-liang Can you provide the corresponding TiDB log? you can use the value of start_ts as a keyword.

Lyle-liang commented 6 years ago

I try again and work well. nothing change.

coocood commented 6 years ago

@Lyle-liang closed for now, feel free to reopen it if the error happens again.