[MapperParsingException[failed to parse [timestamp]] - error.

[timukas]

Hi,

I'm running lates 1.2 snapshot (graylog-1.2.0-SNAPSHOT-20150714073202 + graylog-web-interface-1.2.0-SNAPSHOT-20150714075226).

As a test i'm sending default apache logs to graylog. All required grok patterns are defined. Test message's timestamp is parsed correctly when i press "Try!" button. My grok pattern is:

\s\[%{HTTPDATE:timestamp;date;dd/MMM/yyyy:HH:mm:ss Z}\]\s

When i save an extractor i get errors in graylog server logs:

2015-07-14 14:49:41,895 ERROR: org.graylog2.indexer.messages.Messages - Failed to index [6] messages. Please check the index error log in your web interface for the reason. Error: failure in bulk execution:
[0]: index [gl2-beta_0], type [message], id [68e3c050-2a1e-11e5-950b-e4d53de65bd1], message [MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [2015-07-14T11:49:40.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "2015-07-14T11:49:40.000Z" is malformed at "T11:49:40.000Z"]; ]
[1]: index [gl2-beta_0], type [message], id [6916de40-2a1e-11e5-950b-e4d53de65bd1], message [MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [2015-07-14T11:49:40.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "2015-07-14T11:49:40.000Z" is malformed at "T11:49:40.000Z"]; ]
[2]: index [gl2-beta_0], type [message], id [6949ae10-2a1e-11e5-950b-e4d53de65bd1], message [MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [2015-07-14T11:49:40.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "2015-07-14T11:49:40.000Z" is malformed at "T11:49:40.000Z"]; ]
[3]: index [gl2-beta_0], type [message], id [697ca4f0-2a1e-11e5-950b-e4d53de65bd1], message [MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [2015-07-14T11:49:41.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "2015-07-14T11:49:41.000Z" is malformed at "T11:49:41.000Z"]; ]
[4]: index [gl2-beta_0], type [message], id [69af9bd0-2a1e-11e5-950b-e4d53de65bd1], message [MapperParsingException[failed to parse [timestamp]]; nested: MapperParsingException[failed to parse date field [2015-07-14T11:49:41.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "2015-07-14T11:49:41.000Z" is malformed at "T11:49:41.000Z"]; ]

And some more logs:

[2015-07-14 14:50:41,892][DEBUG][action.bulk              ] [baza] [gl2-beta_0][0] failed to execute bulk item (index) index {[gl2-beta_deflector][message][8d7e5380-2a1e-11e5-950b-e4d53de65bd1], source[{"TIME":"14:50:41","SECOND":"41","gl2_source_node":"6e7bd8a6-ff9f-4b69-9d0e-457c153f5c21","YEAR":"2015","gl2_remote_port":50102,"gl2_remote_ip":"127.0.0.1","timestamp":"2015-07-14T11:50:41.000Z","message":"139.187.243.133 - - [14/Jul/2015:14:50:41 +0300] \"GET /test.php HTTP/1.1\" 200 498 \"-\" \"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)\"","INT":"+0300","_id":"8d7e5380-2a1e-11e5-950b-e4d53de65bd1","source":"127.0.0.1","gl2_source_input":"55a4ed2b44ae96ded64dfeb8","MINUTE":"50","HOUR":"14","streams":[],"MONTH":"Jul","MONTHDAY":"14"}]}
org.elasticsearch.index.mapper.MapperParsingException: failed to parse [timestamp]
    at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:409)
    at org.elasticsearch.index.mapper.object.ObjectMapper.serializeValue(ObjectMapper.java:706)
    at org.elasticsearch.index.mapper.object.ObjectMapper.parse(ObjectMapper.java:497)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:544)
    at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:493)
    at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:480)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:423)
    at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:149)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.performOnPrimary(TransportShardReplicationOperationAction.java:515)
    at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$1.run(TransportShardReplicationOperationAction.java:422)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [2015-07-14T11:50:41.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []
    at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:619)
    at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:547)
    at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:236)
    at org.elasticsearch.index.mapper.core.AbstractFieldMapper.parse(AbstractFieldMapper.java:399)
    ... 12 more
Caused by: java.lang.IllegalArgumentException: Invalid format: "2015-07-14T11:50:41.000Z" is malformed at "T11:50:41.000Z"
    at org.elasticsearch.common.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187)
    at org.elasticsearch.common.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780)
    at org.elasticsearch.index.mapper.core.DateFieldMapper.parseStringValue(DateFieldMapper.java:613)
    ... 15 more

Any ideas of that?

[joschi]

timestamp is an "internal" field of GELF messages (see https://www.graylog.org/resources/gelf/) and shouldn't be overwritten.

If you absolutely want to overwrite the timestamp attribute in your messages, you'll have to use the correct date format (see https://github.com/Graylog2/graylog2-server/blob/2fd51fc472192e7491c692f31448f884d8116fd2/graylog2-plugin-interfaces/src/main/java/org/graylog2/plugin/Tools.java#L65).

[timukas]

Now i'm lost.

If a create regex extractor for apache's date-time field (same logs are used as in above example), name it as a "timestamp", set date type converter to "dd/MMM/yyyy:HH:mm:ss Z", then all works fine.

[joschi]

Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to parse date field [2015-07-14T11:50:41.000Z], tried both date format [yyyy-MM-dd HH:mm:ss.SSS], and timestamp number with locale []

The correct timestamp for this date format would be "2015-07-14 11:50:41.000" (no 'T' between date and time, and no timezone).

[xuanyuanaosheng]

@timukas I have encountered such a problem. the environment is:

• graylog-server v1.3.3
• graylog-web-interfac v1.3.3
• (Oracle Corporation 1.8.0_65 / Linux 2.6.32-573.12.1.el6.x86_64) on centos I have install the Content Pack, https://github.com/Graylog2/graylog-contentpack-nginx , it was used for collect nginx log. It has the same error as yours. I want to know how to solve this problem ？ thanks.