Error occurred during concurrently executing `INSERT INTO ... ON DUPLICATE KEY UPDATE ...`.

Please answer these questions before submitting your issue. Thanks!

What did you do? If possible, provide a recipe for reproducing the error.

使用4个线程并行使用 INSERT INTO ... ON DUPLICATE KEY UPDATE ... 使用spring jdbcTemplate.batchUpdate批量插入，批量执行每次1万条（出现错误次数多些）或1千条（出现错误次数少些，12月12日凌晨3点才出现一次）都出现类似错误。

集群结构： 2个tidb，3个pd，3个tikv

出错信息类似如下：

Caused by: java.sql.BatchUpdateException: Unknown column 'T1' in 'where clause'

nested exception is java.sql.BatchUpdateException: Unknown column 'T2' in 'where clause'

exception is java.sql.BatchUpdateException: Unknown column 'T0' in 'where clause'

报错的sql语句单独执行是正确的。

表结构如下

use zabbix; DROP TABLE IF EXISTS cdn_cache_log; CREATE TABLE cdn_cache_log ( id int auto_increment NOT NULL, key_unique_data varchar(250) NOT NULL, _year int NOT NULL,/ 年 / _month int NOT NULL,/ 月 / _day int NOT NULL,/ 日/ _hour int NOT NULL,/ 时 / _host char(16) NOT NULL,/ 主机地址 /

hua_dan_code char(32) NOT NULL,/ 话单代码例如 010 / hua_dan_name char(32) NOT NULL,/ 话单名称例如 huawei / hua_dan_type char(32) NOT NULL,/ 话单类型例如 ott cdn webcache等 /

domain char(32) NOT NULL,/ 顶级域名 / sub_domain char(64) NOT NULL,/ 子域名 /

tc_request bigint NOT NULL,/ 访问总次数 / t_dl_bytes bigint NOT NULL,/ 访问总流量除以response_time（响应总时长）得到平均下载速率/ response_time bigint NOT NULL,/ 响应总时长除以total_count得到平均响应时延 / response_time_first_byte bigint NOT NULL,/ 首字节响应时长除以total_count得到首字节平均响应时长 / tc_hit_2xx bigint NOT NULL,/ 命中2XX的次数 / tc_hit_3xx bigint NOT NULL,/ 命中3XX的次数 / tc_hit_4xx bigint NOT NULL,/ 命中4XX的次数 / tc_hit_5xx bigint NOT NULL,/ 命中5XX的次数 /

bs_total_count bigint NOT NULL,/ 回源访问总次数 / bs_total_flow bigint NOT NULL,/ 回源访问总流量除以 bs_response_time （回源响应总时长）得到回源访问平均下载速率 / bs_response_time bigint NOT NULL,/ 回源响应总时长 / bs_response_time_first_byte bigint NOT NULL,/ 回源首字节响应时间，除以bs_total_count得到平均首字节时延 / tc_miss_2xx bigint NOT NULL,/ 回源2XX次数 / tc_miss_3xx bigint NOT NULL,/ 回源3XX次数 / tc_miss_4xx bigint NOT NULL,/ 回源4XX次数 / tc_miss_5xx bigint NOT NULL,/ 回源5XX次数 /

ins_date datetime NOT NULL,/ / PRIMARY KEY (id), UNIQUE KEY UNIQUE_KEY_1 (key_unique_data), KEY KEY1 (_year, _month, _day, _hour, domain, sub_domain, hua_dan_code,hua_dan_type) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;

What did you expect to see? 数据完整正确插入表中
What did you see instead?

出错信息：

Tue Dec 12 03:24:29 CST 2017 线程 4 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 158 本线程已经总共执行Sql 37477 所有线程已经批量执行sql语句 124714 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_08_23_010_zte_webcache_114.114.29.43_domain_cache.family.migu.cn', 2017, 12, 08, 23, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'cache.family.migu.cn', 3, 332189, 0, 1402, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:24:26') ON DUPLICATE KEY UPDATE tc_request = tc_request + 3, t_dl_bytes = t_dl_bytes + 332189, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 1402, tc_hit_2xx = tc_hit_2xx + 3, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; nested exception is java.sql.BatchUpdateException: Unknown column 'T1' in 'where clause' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:585) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: Unknown column 'T1' in 'where clause' at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:24:36 CST 2017 线程 1 本线程此次执行Sql 1002 本线程已经总共执行Sql 27424 所有线程已经批量执行sql语句 125716 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:25:24 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:25:24 CST 2017 Tue Dec 12 03:25:24 CST 2017redis 队列为空，本次最终存入bigqueue数据 834 启动后已经总共存入bigqueue数据 126532 Tue Dec 12 03:25:24 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:25:24 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:25:30 CST 2017 线程 4 批处理插入数据库执行开始 Tue Dec 12 03:25:30 CST 2017 Tue Dec 12 03:25:30 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:25:30 CST 2017 Tue Dec 12 03:25:34 CST 2017 线程 2 批处理插入数据库执行结束 Tue Dec 12 03:25:34 CST 2017 Tue Dec 12 03:25:34 CST 2017 线程 2 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 384 本线程已经总共执行Sql 27456 所有线程已经批量执行sql语句 126100 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_01_010_zte_webcache_114.114.29.43_domain_tyst.migu.cn', 2017, 12, 09, 01, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 134, 986529829, 333000, 259416, 126, 0, 7, 1, 13, 49924972, 11000, 7633, 5, 0, 7, 1, '2017-12-12 03:25:30') ON DUPLICATE KEY UPDATE tc_request = tc_request + 134, t_dl_bytes = t_dl_bytes + 986529829, response_time = response_time + 333000, response_time_first_byte = response_time_first_byte + 259416, tc_hit_2xx = tc_hit_2xx + 126, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 7, tc_hit_5xx = tc_hit_5xx + 1, bs_total_count = bs_total_count + 13, bs_total_flow = bs_total_flow + 49924972, bs_response_time = bs_response_time + 11000, bs_response_time_first_byte = bs_response_time_first_byte + 7633, tc_miss_2xx = tc_miss_2xx + 5, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 7, tc_miss_5xx = tc_miss_5xx + 1]; SQL state [HY000]; error code [1105]; line 1 column 471 near "pub, ic, ri, gm, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:25:29') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218); nested exception is java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, ri, gm, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:25:29') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, ri, gm, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:25:29') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:25:36 CST 2017 线程 4 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 452 本线程已经总共执行Sql 37929 所有线程已经批量执行sql语句 126552 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:26:24 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:26:24 CST 2017 Tue Dec 12 03:26:25 CST 2017redis 队列为空，本次最终存入bigqueue数据 997 启动后已经总共存入bigqueue数据 127529 Tue Dec 12 03:26:25 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:26:25 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:26:35 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:26:35 CST 2017 Tue Dec 12 03:26:45 CST 2017 线程 2 批处理插入数据库执行结束 Tue Dec 12 03:26:45 CST 2017 Tue Dec 12 03:26:45 CST 2017 线程 2 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 997 本线程已经总共执行Sql 28453 所有线程已经批量执行sql语句 127549 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:27:25 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:27:25 CST 2017 Tue Dec 12 03:27:25 CST 2017 本次已经存入bigqueue数据 1000 启动后已经总共存入bigqueue数据 128529 Tue Dec 12 03:27:25 CST 2017redis 队列为空，本次最终存入bigqueue数据 1035 启动后已经总共存入bigqueue数据 128564 Tue Dec 12 03:27:25 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:27:25 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:27:37 CST 2017 线程 4 批处理插入数据库执行开始 Tue Dec 12 03:27:37 CST 2017 Tue Dec 12 03:27:37 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:27:37 CST 2017 Tue Dec 12 03:27:43 CST 2017 线程 4 批处理插入数据库执行结束 Tue Dec 12 03:27:43 CST 2017 Tue Dec 12 03:27:43 CST 2017 线程 4 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 559 本线程已经总共执行Sql 38488 所有线程已经批量执行sql语句 128108 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:27:43 CST 2017 线程 1 批处理插入数据库执行结束 Tue Dec 12 03:27:43 CST 2017 Tue Dec 12 03:27:43 CST 2017 线程 1 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 476 本线程已经总共执行Sql 27900 所有线程已经批量执行sql语句 128584 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:28:25 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:28:25 CST 2017 Tue Dec 12 03:28:25 CST 2017redis 队列为空，本次最终存入bigqueue数据 799 启动后已经总共存入bigqueue数据 129363 Tue Dec 12 03:28:25 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:28:25 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:28:43 CST 2017 线程 4 批处理插入数据库执行开始 Tue Dec 12 03:28:43 CST 2017 Tue Dec 12 03:28:43 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:28:43 CST 2017 Tue Dec 12 03:28:48 CST 2017 线程 4 批处理插入数据库执行结束 Tue Dec 12 03:28:48 CST 2017 Tue Dec 12 03:28:48 CST 2017 线程 4 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 376 本线程已经总共执行Sql 38864 所有线程已经批量执行sql语句 128960 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_06_010_zte_webcache_114.114.29.43_domain_114.215.207.183', 2017, 12, 09, 06, '114.114.29.43', '010', 'zte', 'webcache', '114.215.207.183', '114.215.207.183', 4, 452, 0, 2536, 4, 0, 0, 0, 4, 452, 0, 2535, 4, 0, 0, 0, '2017-12-12 03:28:43') ON DUPLICATE KEY UPDATE tc_request = tc_request + 4, t_dl_bytes = t_dl_bytes + 452, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 2536, tc_hit_2xx = tc_hit_2xx + 4, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 4, bs_total_flow = bs_total_flow + 452, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 2535, tc_miss_2xx = tc_miss_2xx + 4, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; nested exception is java.sql.BatchUpdateException: Unknown column 'T2' in 'where clause' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: Unknown column 'T2' in 'where clause' at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:28:49 CST 2017 线程 1 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 425 本线程已经总共执行Sql 28325 所有线程已经批量执行sql语句 129385 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:29:25 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:29:25 CST 2017 Tue Dec 12 03:29:25 CST 2017 本次已经存入bigqueue数据 1000 启动后已经总共存入bigqueue数据 130363 Tue Dec 12 03:29:25 CST 2017redis 队列为空，本次最终存入bigqueue数据 1204 启动后已经总共存入bigqueue数据 130567 Tue Dec 12 03:29:25 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:29:25 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:29:46 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:29:46 CST 2017 Tue Dec 12 03:29:49 CST 2017 线程 4 批处理插入数据库执行开始 Tue Dec 12 03:29:49 CST 2017 Tue Dec 12 03:29:52 CST 2017 线程 4 批处理插入数据库执行结束 Tue Dec 12 03:29:52 CST 2017 Tue Dec 12 03:29:52 CST 2017 线程 4 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 204 本线程已经总共执行Sql 39068 所有线程已经批量执行sql语句 129589 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:29:56 CST 2017 线程 2 批处理插入数据库执行结束 Tue Dec 12 03:29:56 CST 2017 Tue Dec 12 03:29:56 CST 2017 线程 2 本线程此次执行Sql 1000 本线程已经总共执行Sql 29453 所有线程已经批量执行sql语句 130589 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:30:25 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:30:25 CST 2017 Tue Dec 12 03:30:26 CST 2017 本次已经存入bigqueue数据 1000 启动后已经总共存入bigqueue数据 131567 Tue Dec 12 03:30:26 CST 2017redis 队列为空，本次最终存入bigqueue数据 1199 启动后已经总共存入bigqueue数据 131766 Tue Dec 12 03:30:26 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:30:26 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:30:27 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:30:27 CST 2017 org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_10_010_zte_webcache_114.114.29.43_domain_img01.12530.com', 2017, 12, 09, 10, '114.114.29.43', '010', 'zte', 'webcache', '12530.com', 'img01.12530.com', 2, 23369, 0, 1002, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:30:27') ON DUPLICATE KEY UPDATE tc_request = tc_request + 2, t_dl_bytes = t_dl_bytes + 23369, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 1002, tc_hit_2xx = tc_hit_2xx + 2, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; nested exception is java.sql.BatchUpdateException: Unknown column 'T0' in 'where clause' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:585) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: Unknown column 'T0' in 'where clause' at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:30:37 CST 2017 线程 2 本线程此次执行Sql 1002 本线程已经总共执行Sql 30455 所有线程已经批量执行sql语句 131591 本地文件队列中等待插入数据条数 199, 开始不停睡眠 30 秒！ Tue Dec 12 03:30:37 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:30:37 CST 2017 Tue Dec 12 03:30:40 CST 2017 线程 2 批处理插入数据库执行结束 Tue Dec 12 03:30:40 CST 2017 Tue Dec 12 03:30:40 CST 2017 线程 2 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 199 本线程已经总共执行Sql 30654 所有线程已经批量执行sql语句 131790 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:31:26 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:31:26 CST 2017 Tue Dec 12 03:31:26 CST 2017redis 队列为空，本次最终存入bigqueue数据 838 启动后已经总共存入bigqueue数据 132604 Tue Dec 12 03:31:26 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:31:26 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:31:41 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:31:41 CST 2017 org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_11_010_zte_webcache_114.114.29.43_domain_vod.migu.cn', 2017, 12, 09, 11, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'vod.migu.cn', 11, 282463811, 93000, 5852, 11, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:31:41') ON DUPLICATE KEY UPDATE tc_request = tc_request + 11, t_dl_bytes = t_dl_bytes + 282463811, response_time = response_time + 93000, response_time_first_byte = response_time_first_byte + 5852, tc_hit_2xx = tc_hit_2xx + 11, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; nested exception is java.sql.BatchUpdateException: Unknown column 'T0' in 'where clause' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: Unknown column 'T0' in 'where clause' at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:31:50 CST 2017 线程 2 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 840 本线程已经总共执行Sql 31494 所有线程已经批量执行sql语句 132630 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:32:26 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:32:26 CST 2017 Tue Dec 12 03:32:26 CST 2017 本次已经存入bigqueue数据 1000 启动后已经总共存入bigqueue数据 133604 Tue Dec 12 03:32:26 CST 2017redis 队列为空，本次最终存入bigqueue数据 1208 启动后已经总共存入bigqueue数据 133812 Tue Dec 12 03:32:26 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:32:26 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:32:49 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:32:49 CST 2017 Tue Dec 12 03:32:50 CST 2017 线程 2 批处理插入数据库执行开始 Tue Dec 12 03:32:50 CST 2017 Tue Dec 12 03:32:53 CST 2017 线程 2 批处理插入数据库执行结束 Tue Dec 12 03:32:53 CST 2017 Tue Dec 12 03:32:53 CST 2017 线程 2 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 208 本线程已经总共执行Sql 31702 所有线程已经批量执行sql语句 132838 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_14_010_zte_webcache_114.114.29.43_domain_img01.12530.com', 2017, 12, 09, 14, '114.114.29.43', '010', 'zte', 'webcache', '12530.com', 'img01.12530.com', 1, 127597, 0, 770, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:32:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 127597, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 770, tc_hit_2xx = tc_hit_2xx + 1, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; SQL state [HY000]; error code [1105]; line 1 column 471 near "pub, ic, pr, du, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:32:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218); nested exception is java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, pr, du, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:32:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:585) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, pr, du, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:32:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:33:00 CST 2017 线程 1 本线程此次执行Sql 1002 本线程已经总共执行Sql 29327 所有线程已经批量执行sql语句 133840 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:33:26 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:33:26 CST 2017 Tue Dec 12 03:33:27 CST 2017redis 队列为空，本次最终存入bigqueue数据 875 启动后已经总共存入bigqueue数据 134687 Tue Dec 12 03:33:27 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:33:27 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:33:30 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:33:30 CST 2017 org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_15_010_zte_webcache_114.114.29.43_domain_acstatic.migu.cn', 2017, 12, 09, 15, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'acstatic.migu.cn', 12, 14496034, 3000, 10470, 12, 0, 0, 0, 8, 2326767, 2000, 5065, 8, 0, 0, 0, '2017-12-12 03:33:30') ON DUPLICATE KEY UPDATE tc_request = tc_request + 12, t_dl_bytes = t_dl_bytes + 14496034, response_time = response_time + 3000, response_time_first_byte = response_time_first_byte + 10470, tc_hit_2xx = tc_hit_2xx + 12, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 8, bs_total_flow = bs_total_flow + 2326767, bs_response_time = bs_response_time + 2000, bs_response_time_first_byte = bs_response_time_first_byte + 5065, tc_miss_2xx = tc_miss_2xx + 8, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; nested exception is java.sql.BatchUpdateException: Unknown column 'mcon' in 'where clause' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: Unknown column 'mcon' in 'where clause' at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:33:39 CST 2017 线程 1 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 877 本线程已经总共执行Sql 30204 所有线程已经批量执行sql语句 134717 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:34:27 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:34:27 CST 2017 Tue Dec 12 03:34:27 CST 2017redis 队列为空，本次最终存入bigqueue数据 936 启动后已经总共存入bigqueue数据 135623 Tue Dec 12 03:34:27 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:34:27 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:34:40 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:34:40 CST 2017 org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_17_010_zte_webcache_114.114.29.43_domain_tyst.migu.cn', 2017, 12, 09, 17, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 357, 411905287, 212000, 304812, 116, 0, 239, 2, 284, 108906332, 114000, 183535, 43, 0, 239, 2, '2017-12-12 03:34:40') ON DUPLICATE KEY UPDATE tc_request = tc_request + 357, t_dl_bytes = t_dl_bytes + 411905287, response_time = response_time + 212000, response_time_first_byte = response_time_first_byte + 304812, tc_hit_2xx = tc_hit_2xx + 116, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 239, tc_hit_5xx = tc_hit_5xx + 2, bs_total_count = bs_total_count + 284, bs_total_flow = bs_total_flow + 108906332, bs_response_time = bs_response_time + 114000, bs_response_time_first_byte = bs_response_time_first_byte + 183535, tc_miss_2xx = tc_miss_2xx + 43, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 239, tc_miss_5xx = tc_miss_5xx + 2]; nested exception is java.sql.BatchUpdateException: Unknown column 'ic' in 'where clause' at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: Unknown column 'ic' in 'where clause' at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:34:49 CST 2017 线程 1 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 938 本线程已经总共执行Sql 31142 所有线程已经批量执行sql语句 135655 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:35:27 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:35:27 CST 2017 Tue Dec 12 03:35:27 CST 2017 本次已经存入bigqueue数据 1000 启动后已经总共存入bigqueue数据 136623 Tue Dec 12 03:35:27 CST 2017redis 队列为空，本次最终存入bigqueue数据 1190 启动后已经总共存入bigqueue数据 136813 Tue Dec 12 03:35:27 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:35:27 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:35:50 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:35:50 CST 2017 Tue Dec 12 03:35:52 CST 2017 线程 4 批处理插入数据库执行开始 Tue Dec 12 03:35:52 CST 2017 org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_19_010_zte_webcache_114.114.29.43_domain_114.215.207.183', 2017, 12, 09, 19, '114.114.29.43', '010', 'zte', 'webcache', '114.215.207.183', '114.215.207.183', 4, 452, 1000, 2708, 4, 0, 0, 0, 4, 452, 1000, 2705, 4, 0, 0, 0, '2017-12-12 03:35:52') ON DUPLICATE KEY UPDATE tc_request = tc_request + 4, t_dl_bytes = t_dl_bytes + 452, response_time = response_time + 1000, response_time_first_byte = response_time_first_byte + 2708, tc_hit_2xx = tc_hit_2xx + 4, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 4, bs_total_flow = bs_total_flow + 452, bs_response_time = bs_response_time + 1000, bs_response_time_first_byte = bs_response_time_first_byte + 2705, tc_miss_2xx = tc_miss_2xx + 4, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; SQL state [HY000]; error code [1105]; line 1 column 482 near " 2/, 9 , '114.114.29.43', '010', 'zte', 'webcache', 'cmvideo.cn', 'wapx.cmvideo.cn', 1, 16227, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:52') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 16227, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 1, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1235); nested exception is java.sql.BatchUpdateException: line 1 column 482 near " 2/, 9 , '114.114.29.43', '010', 'zte', 'webcache', 'cmvideo.cn', 'wapx.cmvideo.cn', 1, 16227, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:52') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 16227, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 1, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1235) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: line 1 column 482 near " 2/, 9 , '114.114.29.43', '010', 'zte', 'webcache', 'cmvideo.cn', 'wapx.cmvideo.cn', 1, 16227, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:52') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 16227, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 1, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1235) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:35:54 CST 2017 线程 4 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 192 本线程已经总共执行Sql 39260 所有线程已经批量执行sql语句 135847 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_19_010_zte_webcache_114.114.29.43_domain_cache.family.migu.cn', 2017, 12, 09, 19, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'cache.family.migu.cn', 9, 791931, 0, 4060, 9, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:50') ON DUPLICATE KEY UPDATE tc_request = tc_request + 9, t_dl_bytes = t_dl_bytes + 791931, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 4060, tc_hit_2xx = tc_hit_2xx + 9, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; SQL state [HY000]; error code [1105]; line 1 column 471 near "pub, ic, ri, gm, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218); nested exception is java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, ri, gm, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:585) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, ri, gm, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:35:49') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:36:00 CST 2017 线程 1 本线程此次执行Sql 1002 本线程已经总共执行Sql 32144 所有线程已经批量执行sql语句 136849 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！ Tue Dec 12 03:36:27 CST 2017 连接redis并获取数据开始 Tue Dec 12 03:36:27 CST 2017 Tue Dec 12 03:36:27 CST 2017redis 队列为空，本次最终存入bigqueue数据 880 启动后已经总共存入bigqueue数据 137693 Tue Dec 12 03:36:27 CST 2017 redis中已经没有数据需要处理，暂时跳出循环 Tue Dec 12 03:36:27 CST 2017 睡眠 60 秒后将再次进行redis数据提取操作！ Tue Dec 12 03:36:31 CST 2017 线程 1 批处理插入数据库执行开始 Tue Dec 12 03:36:31 CST 2017 org.springframework.jdbc.UncategorizedSQLException: StatementCallback; uncategorized SQLException for SQL [INSERT INTO cdn_cache_log (key_unique_data, _year, _month, _day, _hour, _host, hua_dan_code, hua_dan_name, hua_dan_type, domain, sub_domain, tc_request, t_dl_bytes, response_time, response_time_first_byte, tc_hit_2xx, tc_hit_3xx, tc_hit_4xx, tc_hit_5xx, bs_total_count, bs_total_flow, bs_response_time, bs_response_time_first_byte, tc_miss_2xx, tc_miss_3xx, tc_miss_4xx, tc_miss_5xx, ins_date) VALUES ('2017_12_09_20_010_zte_webcache_114.114.29.43_domain_mcontent.10086.cn', 2017, 12, 09, 20, '114.114.29.43', '010', 'zte', 'webcache', '10086.cn', 'mcontent.10086.cn', 15, 634723, 0, 8083, 15, 0, 0, 0, 9, 497961, 0, 4852, 9, 0, 0, 0, '2017-12-12 03:36:31') ON DUPLICATE KEY UPDATE tc_request = tc_request + 15, t_dl_bytes = t_dl_bytes + 634723, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 8083, tc_hit_2xx = tc_hit_2xx + 15, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 9, bs_total_flow = bs_total_flow + 497961, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 4852, tc_miss_2xx = tc_miss_2xx + 9, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0]; SQL state [HY000]; error code [1105]; line 1 column 471 near "pub, ic, pr, du, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:36:30') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218); nested exception is java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, pr, du, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:36:30') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:83) at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:407) at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:555) at storm.project.bolt.tools.ProjectCdnStormCountDataIntoMySQL$ThreadDataSaveToMySQL.run(ProjectCdnStormCountDataIntoMySQL.java:497) at java.lang.Thread.run(Thread.java:748) Caused by: java.sql.BatchUpdateException: line 1 column 471 near "pub, ic, pr, du, '114.114.29.43', '010', 'zte', 'webcache', 'migu.cn', 'tyst.migu.cn', 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, '2017-12-12 03:36:30') ON DUPLICATE KEY UPDATE tc_request = tc_request + 1, t_dl_bytes = t_dl_bytes + 0, response_time = response_time + 0, response_time_first_byte = response_time_first_byte + 0, tc_hit_2xx = tc_hit_2xx + 0, tc_hit_3xx = tc_hit_3xx + 0, tc_hit_4xx = tc_hit_4xx + 0, tc_hit_5xx = tc_hit_5xx + 0, bs_total_count = bs_total_count + 0, bs_total_flow = bs_total_flow + 0, bs_response_time = bs_response_time + 0, bs_response_time_first_byte = bs_response_time_first_byte + 0, tc_miss_2xx = tc_miss_2xx + 0, tc_miss_3xx = tc_miss_3xx + 0, tc_miss_4xx = tc_miss_4xx + 0, tc_miss_5xx = tc_miss_5xx + 0" (total length 1218) at com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:1139) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:536) at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:1) at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396) ... 3 more Tue Dec 12 03:36:40 CST 2017 线程 1 本地文件队列为空，没有数据需要插入mysql，本线程此次执行Sql 882 本线程已经总共执行Sql 33026 所有线程已经批量执行sql语句 137731 本地文件队列中等待插入数据条数 0, 开始不停睡眠 30 秒！

What version of TiDB are you using (tidb-server -V)?

Release Version: v1.0.2-16-gb8bcf9c Git Commit Hash: b8bcf9c3d68dc16e2d75a99212686130f895b748 Git Commit Branch: release-1.0 UTC Build Time: 2017-11-25 07:55:49

pingcap / tidb

Error occurred during concurrently executing `INSERT INTO ... ON DUPLICATE KEY UPDATE ...`. #5376