search

konklone / oversight.garden

Bringing together the oversight community's work.
https://oversight.garden
Creative Commons Zero v1.0 Universal
26 stars 9 forks source link

Request size exceeded 10485760 bytes #199

Closed divergentdave closed 7 years ago

divergentdave commented 7 years ago

While ingesting GAO reports, I got the following error:

{"Message":"Request size exceeded 10485760 bytes"}
divergentdave commented 7 years ago

Ugh, this is an AWS-imposed limit on their Elasticsearch service. Some instance sizes have limits of 10MB, while other instance sizes have limits of 100MB. https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/aes-limits.html

divergentdave commented 7 years ago

There's only one report that exceeds this, I may just monkey with it by hand or truncate it and move on for now. The text appears to be a messy OCR of a 1200 page document. -rw-rw-r-- 1 ubuntu ubuntu 12514362 Jul 20 02:48 data/gaoreports/1980/PAD-80-50/ report.txt