[Bug] Elasticsearch read timeouts occurring in twitter_monitor, embedder, and reindex.

TheRensselaerIDEA / twitter-nlp

Data Analytics on Twitter with Natural Language Processing

MIT License

17 stars 7 forks source link

[Bug] Elasticsearch read timeouts occurring in twitter_monitor, embedder, and reindex. #9

Closed AbrahamSanders closed 4 years ago

AbrahamSanders commented 4 years ago

Fix directly by increasing client default timeout to a higher (configurable) value.

Also, optimize twitter_monitor by reducing chattiness with Elasticsearch - instead of indexing each new tweet in a separate request, collect them in a bucket and make a bulk index request when the bucket is full.

AbrahamSanders commented 4 years ago

Fixed by 8a24dd1d9247f22000a87ab0f3eacd7a276bca35