influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.5k stars 5.55k forks source link

my regex is corrected,but the log parser is not run the regex to influxdb #2495

Closed jokerbo1 closed 7 years ago

jokerbo1 commented 7 years ago

Hello , I found a problem ,it's my telgraf run the log-parser plugin don't run the regex to influxdb, this my custom partten; custom_patterns = ''' VRHT %{WORD} %{URIPATHPARAM} HTTP/%{NUMBER} NEWHREFER (%{URI}|-)? NGINX_ACCESSPA %{IPV4:remote_addr}|%{NGUSER:remote_user}|[%{HTTPDATE:time_local}]|%{URIHOST:host}|%{VRHT:request}|%{INT :status}|%{NUMBER:body_bytes_sent}|%{NEWHREFER:http_referer}|%{DATA:http_user_agent}|%{NGUSER:http_x_forwarded_for}|%{NUMBER:request_time}|%{NUMBER:upstream_response_time}|%{NUMBER:upstream_connect_time}|%{NUMBER:upstream_header_time}|%{DATA:upstream_http_via}|%{GREEDYDATA:upstream_addr}|%{GREEDYDATA:upstream_http_x_e_reqid}|%{GREEDYDATA:upstream_http_x_m_reqid} ''' I'm debug by the http://grokdebug.herokuapp.com/patterns# is pass ,the partten is used by this log format : 10.27.2.12|-|[10/Jan/2017:13:58:29 +0800]|file.battleofballs.com|GET /27728639_1472616368_3504720.jpeg HTTP/1.1|200|0|-|Dalvik/2.1.0 (Linux; U; Android 5.1.1; HUAWEI P8max Build/HUAWEIDAV-703L)|-|0.001|0.001|0.000|0.001|http/1.1 cdn-cnc-gddg-dg-6 (ApacheTrafficServer/6.2.0 [cHs f ])|127.0.0.1:8080|1484027909022684633:25397058|UjMAAFnCJfhJTJgU

service telegraf start ,view the log all prompt ; 2017/03/06 12:02:10 Output [influxdb] buffer fullness: 0 / 100000 metrics. Total gathered metrics: 0. Total dropped metrics: 0. 2017/03/06 12:02:20 Input [logparser] gathered metrics, (10s interval) in 26.557µs 2017/03/06 12:02:20 Output [influxdb] buffer fullness: 0 / 100000 metrics. Total gathered metrics: 0. Total dropped metrics: 0. 2017/03/06 12:02:30 Input [logparser] gathered metrics, (10s interval) in 20.468µs

i'm feeling the plugin is not run my regex ,but i'not find the errorlog ,can you help me? Thanks.

danielnelson commented 7 years ago

I think you need to escape special characters in your pattern, such as in this snippet:

[inputs.logparser.grok]
  patterns = ["%{IPV4:remote_addr}\\|%{NGUSER:remote_user}\\|\\[%{HTTPDATE:time_local}\\]\\|%{URIHOST:host}\\|%{VRHT:request}\\|%{INT:status}\\|%{NUMBER:body_bytes_sent}\\|%{NEWHREFER:http_referer}\\|%{DATA:http_user_agent}\\|%{NGUSER:http_x_forwarded_for}\\|%{NUMBER:request_time}\\|%{NUMBER:upstream_response_time}\\|%{NUMBER:upstream_connect_time}\\|%{NUMBER:upstream_header_time}\\|%{DATA:upstream_http_via}\\|%{GREEDYDATA:upstream_addr}\\|%{GREEDYDATA:upstream_http_x_e_reqid}\\|%{GREEDYDATA:upstream_http_x_m_reqid}"]
  custom_patterns = '''
    VRHT %{WORD} %{URIPATHPARAM} HTTP/%{NUMBER}
    NEWHREFER (%{URI}|-)?
  '''
jokerbo1 commented 7 years ago

Thanks .