CircuitBreakingException: Data too large

[shcherbak]

Elasticsearch version: 5.2.1

Plugins installed: [No]

JVM version (java -version): java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1.8.0_121-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

OS version (uname -a if on a Unix-like system): Linux fo-sys-els-02.srv.icecat.biz 4.9.0-0.bpo.2-amd64 #1 SMP Debian 4.9.13-1~bpo8+1 (2017-02-27) x86_64 GNU/Linux

Description of the problem including expected versus actual behavior: cluster fails on date parsing bigger then java.Date max value

Steps to reproduce:

1. add message with big date field for example '2050-01-01'

Provide logs (if relevant): 2017-05-29T12:13:17,482][DEBUG][o.e.a.b.TransportShardBulkAction] [els-02] [product_multimedia_object][3] failed to execute bulk item (index) index {[product_multimedia_object][product_multimedia_objecttype][22573736|5534367], source[{"product_id":"22573736","updated":1495030982,"data_source_id":"0","size":"6987938","md5_origin":"94b5057c05c087a89d706fcfc0c8e875","uuid":"bb8b2cc1-756b-48e7-9f20-430154832096","link":29272.pdf","short_descr":"Building instructions","link_origin":"","id":"5534367","langid":"1","type":"pdf","ExpiryDate":"2050-12-12","content_type":"application\/pdf"}]} org.elasticsearch.index.mapper.MapperParsingException: failed to parse [ExpiryDate] at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:298) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:449) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:575) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:395) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:372) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:93) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:66) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:275) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:533) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:510) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:196) ~[elasticsearch-5.2.1.jar:5.2.1] [2017-05-30T08:00:02,930][WARN ][r.suppressed ] path: /product_multimedia_object/product_multimedia_objecttype/_search, params: {scroll=10s, index=product_multimedia_object, type=product_multimedia_objecttype} org.elasticsearch.common.breaker.CircuitBreakingException: [parent] Data too large, data for [] would be larger than limit of [1951531007/1.8gb] at org.elasticsearch.indices.breaker.HierarchyCircuitBreakerService.checkParentLimit(HierarchyCircuitBreakerService.java:211) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.addEstimateBytesAndMaybeBreak(ChildMemoryCircuitBreaker.java:128) ~[elasticsearch-5.2.1.jar:5.2.1] at org.elasticsearch.http.HttpServer.dispatchRequest(HttpServer.java:109) [elasticsearch-5.2.1.jar:5.2.1]

[cbuescher]

This doesn't look like a date parsing problem to me. Adding a simple document like {"ExpiryDate":"2050-12-12"} works just fine for me in 5.2.1. @shcherbak where do you see the "java data parse error"? What's the mappings for the "ExpiryDate" field? Also the source of the document in the log looks strange, my json formatter complains starting with ..."link":29272.pdf"... where the quoting of the string seems to be broken somehow. How do you generate those documents?

[shcherbak]

@cbuescher , here it is: org.elasticsearch.index.mapper.MapperParsingException: failed to parse [ExpiryDate] "link":29272.pdf" - vas a valid json, i'v deleted some of data because of security issue (domain, resource etc). it sould be "link":"29272.pdf"

[cbuescher]

Ok, thanks for clearing up the issue with the documents "source". As I mentioned, the error shown doesn't point to a problem with the date parsing, it only happens with your "ExpiryDate" field. I checked that there is nothing wrong in general with the format, without a mapping "2050-12-12" gets recognized as a date and parsed correctly. So in order to reproduce this we need a minimal example of a mapping and document that show this behaviour, otherwise I suspect some other issue with your bulk request or your mapping.

[shcherbak]

@cbuescher , i have reproduce (100%)

`GET product_gallery/product_gallerytype/_search { "query": { "match": { "_id": "21213533" } } }

{ "took": 264, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 1, "hits": [ { "_index": "product_gallery", "_type": "product_gallerytype", "_id": "21213533", "_score": 1, "_source": { "product_id": "37033502", "low_height": "200", "id": "21213533", "low_width": "102", "link_origin": "link_origin_masked/S450_222-02.jpg", "medium_size": "29661", "data_source_id": "0", "editor_user_id": "52979", "ExpiryDate": "2050-01-01 00:00:00", "medium_height": "500", "md5_origin": "c0131ddba6aba7431636f775af01933b", "link_raw": "link_raw_masked/37033502_7964155893.jpeg", "medium_width": "256", "thumb_size": "923", "is_main": true, "low_link": "low_link_masked/37033502_7964155893.jpg", "thumb_link": "thumb_link_masked/37033502_7964155893.jpg", "height": "1772", "low_size": "7573", "quality": "1", "medium_link": "medium_link_masked/37033502_7964155893.jpg", "langid": "0", "width": "909", "link": "link_masked/37033502_7964155893.jpg", "size": "98721", "size_origin": "49105", "updated": "2050-01-01 00:00:00", "no": "2" } } ] } }`

buffer overflows and then we see in logs: Data too large, data for [] would be larger than limit of [195153100 7/1.8gb]

there is no data such size, even indexes are smaller. this is a test system

[cbuescher]

So just to be clear, this is not a date parsing error. I updated the title accordingly. I think your issue might be related to a bug fixed in https://github.com/elastic/elasticsearch/pull/23310 which should be in 5.2.2. I'd advise you try this with a version higher than 5.2.1.

[colings86]

No further feedback. If you manage to reproduce this on 5.2.2+ please reopen this issue and provide details