search

lucene-cn / lxdb

14 stars 2 forks source link

kafka解析出错 日志要抽样打印 #127

Open muyannian opened 3 years ago

muyannian commented 3 years ago

2-24 15:45:20.292] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.293] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,c7eea6e28241a12db0731244cf646029, java.lang.NumberFormatException: For input string: "c7eea6e28241a12db0731244cf646029" [2021-02-24 15:45:20.293] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.294] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,894a595067ab3507bed3da9c6235ac6d, java.lang.NumberFormatException: For input string: "894a595067ab3507bed3da9c6235ac6d" [2021-02-24 15:45:20.294] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.294] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,4f5c78e176e06db7aa8d03a0eaab3ee2, java.lang.NumberFormatException: For input string: "4f5c78e176e06db7aa8d03a0eaab3ee2" [2021-02-24 15:45:20.294] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.295] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,0234b03de93d52065807543f786c69db, java.lang.NumberFormatException: For input string: "0234b03de93d52065807543f786c69db" [2021-02-24 15:45:20.295] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.296] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,30af514ac3f272b2e0766573a3990583, java.lang.NumberFormatException: For input string: "30af514ac3f272b2e0766573a3990583" [2021-02-24 15:45:20.296] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.296] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,2ca28197f3d691111c7cbf91d19a0005, java.lang.NumberFormatException: For input string: "2ca28197f3d691111c7cbf91d19a0005" [2021-02-24 15:45:20.296] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.297] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,8615b6432d1d5f91b7fa73f560bc9308, java.lang.NumberFormatException: For input string: "8615b6432d1d5f91b7fa73f560bc9308" [2021-02-24 15:45:20.297] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.298] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,6f60b0c0666320e6d40dac3d648ce8ea, java.lang.NumberFormatException: For input string: "6f60b0c0666320e6d40dac3d648ce8ea" [2021-02-24 15:45:20.298] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.299] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,dcc2f58d12816d05cecc53cc3772bbbf, java.lang.NumberFormatException: For input string: "dcc2f58d12816d05cecc53cc3772bbbf" [2021-02-24 15:45:20.299] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.300] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,4a76a683b63b72ad4da3b98ba6979e1d, java.lang.NumberFormatException: For input string: "4a76a683b63b72ad4da3b98ba6979e1d" [2021-02-24 15:45:20.300] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.301] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,6b76c290fdaebfab872327fd7f6a6a47, java.lang.NumberFormatException: For input string: "6b76c290fdaebfab872327fd7f6a6a47" [2021-02-24 15:45:20.301] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.301] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,9beb991e839773965b79ea04a3bac0c3, java.lang.NumberFormatException: For input string: "9beb991e839773965b79ea04a3bac0c3" [2021-02-24 15:45:20.301] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.302] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,253cc6d137a32dc1fc9f9c1d11dd3c52, java.lang.NumberFormatException: For input string: "253cc6d137a32dc1fc9f9c1d11dd3c52" [2021-02-24 15:45:20.302] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.303] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,54ea1abb1e2a80553b8611c0a1dd9907, java.lang.NumberFormatException: For input string: "54ea1abb1e2a80553b8611c0a1dd9907" [2021-02-24 15:45:20.303] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.304] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,3cf19548a58363f5260d29c14bb6fc14, java.lang.NumberFormatException: For input string: "3cf19548a58363f5260d29c14bb6fc14" [2021-02-24 15:45:20.304] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.305] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,3ffd843f5d01e0f53cff1e5b75017ce9, java.lang.NumberFormatException: For input string: "3ffd843f5d01e0f53cff1e5b75017ce9" [2021-02-24 15:45:20.305] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.306] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,0361a207027fee59d8de7cd74c618064, java.lang.NumberFormatException: For input string: "0361a207027fee59d8de7cd74c618064" [2021-02-24 15:45:20.306] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.306] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,12e5f44acb2021069bd0f3ff5ef7c226, java.lang.NumberFormatException: For input string: "12e5f44acb2021069bd0f3ff5ef7c226" [2021-02-24 15:45:20.307] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.309] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,a52fbe9e386daff2dc9592153294b6f1, java.lang.NumberFormatException: For input string: "a52fbe9e386daff2dc9592153294b6f1" [2021-02-24 15:45:20.309] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.309] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,a935a1ddcb440ccff5d1164c552acd97, java.lang.NumberFormatException: For input string: "a935a1ddcb440ccff5d1164c552acd97" [2021-02-24 15:45:20.309] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.310] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,d03ebf0c2b40772676e60c0c7699a399, java.lang.NumberFormatException: For input string: "d03ebf0c2b40772676e60c0c7699a399" [2021-02-24 15:45:20.310] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.310] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,c027856f2140d33519c0ed5196f3d3a1, java.lang.NumberFormatException: For input string: "c027856f2140d33519c0ed5196f3d3a1" [2021-02-24 15:45:20.311] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.311] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,5ae85bd362467c6e28ec026c2342a0e6, java.lang.NumberFormatException: For input string: "5ae85bd362467c6e28ec026c2342a0e6" [2021-02-24 15:45:20.311] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Illegal columnName: primarykey [2021-02-24 15:45:20.312] (com.luxin.hydrogen.stream.HydrogenKafkaConsumer:705)[ERROR] - Kafka Parse Illegal value class: integer,id,496a5d74500144ea1c0bed0a327a10f4, java.lang.NumberFormatException: For input string: "496a5d74500144ea1c0bed0a327a10f4"

muyannian commented 3 years ago 
// 相同错误日志仅打印一次，避免慢日志现象
private void printOneError(String errorMsg) {
    if (!lastErrorMsg.equals(errorMsg)) {
        lastErrorMsg = errorMsg;
        LOG.error(errorMsg);
    }
}

添加时间间隔判断