influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.9k stars 5.6k forks source link

field corresponding to `timezone' is not defined in `*grok.Parser' #3020

Closed zhugw closed 7 years ago

zhugw commented 7 years ago

My log is like this

2017-07-15 14:57:27.796 [http-nio-8080-exec-1] INFO  c.i.s.controller.LogInterceptor c4b496f0185e43e0b433eda8f5ec7c73 - ip: 0:0:0:0:0:0:0:1 uri: /search queryString: col1s=foo&cols=bar&level=-1&start=2017-06-12+00%3A00&end=2017-07-10+15%3A00

My customer pattern

CUSTOM_LOG_1 %{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} \[http-nio-%{POSINT}-exec-%{INT}\] INFO  (?:[a-zA-Z0-9-]+\.)+[A-Za-z0-9$]+ %{WORD:reqId} - ip: %{IP:ip} uri: %{URIPATH:uri} queryString: %{GREEDYDATA:queryString}

but actually I got time is 1500130647796000000(Sat Jul 15 22:57:27 CST 2017) and actually I want is 2017-07-15 14:57:27

test_log_1,host=localhost reqId="c4b496f0185e43e0b433eda8f5ec7c73",ip="0:0:0:0:0:0:0:1",uri="/search",queryString="col1s=foo&cols=bar&level=-1&start=2017-06-12+00%3A00&end=2017-07-10+15%3A00" 1500130647796000000

So I explicitly add timezone within inputs.logparser.grok as README described

  [inputs.logparser.grok]
    patterns = ["%{CUSTOM_LOG_1}"]
    custom_patterns = '''
      CUSTOM_LOG_1 %{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} \[http-nio-%{POSINT}-exec-%{INT}\] INFO  (?:[a-zA-Z0-9-]+\.)+[A-Za-z0-9$]+ %{WORD:reqId} - ip: %{IP:ip} uri: %{URIPATH:uri} queryString: %{GREEDYDATA:queryString}
    '''
    timezone = "Local"

but I got below error

2017/07/15 16:47:38 E! Error parsing /tmp/telegraf.conf, line 14: field corresponding to `timezone' is not defined in `*grok.Parser'

My telegraf version is

➜  ~ telegraf --version
Telegraf v1.3.3 (git: unknown unknown)
gibsonc commented 7 years ago

I had the same issue but got one step further. Looking at sample configuration in logparser.go, I see the timezone option needs to be inside the custom_patterns declaration.

  custom_patterns = '''

    timezone = "Canada/Eastern"
    '''

That said, no matter what I do to timezone, even setting it to rubbish, I don't get the desired result.

danielnelson commented 7 years ago

This feature was only added for the upcoming Telegraf 1.4.

The docs.influxdata.com page currently links to the latest development documentation, sorry about the confusion. I have an issue opened to address this here: https://github.com/influxdata/docs.influxdata.com/issues/1195