ETL does not insert records into cluster

mozilla / Bugzilla-ETL

ETL for feeding bug snapshots to an ElasticSearch cluster

Mozilla Public License 2.0

5 stars 9 forks source link

ETL does not insert records into cluster #11

Closed klahnakoski closed 6 years ago

klahnakoski commented 6 years ago

There is a chance that records will not go into the ES cluster if the cluster is taking too long to respond. The threaded queue is notified of shutdown with __exit()__, and it does not ensure he records are flushed.

klahnakoski commented 6 years ago

Observed here: https://bugzilla.mozilla.org/show_bug.cgi?id=1494959

klahnakoski commented 6 years ago

The ETL is fine, I was looking at the dev machine. 😢

The problem was logic in the dashboard https://github.com/klahnakoski/FreshOranges/commit/daa033c53bd453421176387d33467f6a7aed0a19?w=1