search

mozilla / ActiveData

Provide high speed filtering and aggregation over data
Mozilla Public License 2.0
31 stars 20 forks source link

Classification data in 'treeherder' table has stopped updating #159

Closed Archaeopteryx closed 5 years ago

Archaeopteryx commented 5 years ago

Classification data in 'treeherder' table has stopped updating, fresh and neglected oranges are empty.

The query from neglected oranges gets maximum 11 classifications for a bug during the last week but it has been classified more often.

The newest bug used for classification is a week old.

jmaher commented 5 years ago

@klahnakoski can you help get this online when you get back on Monday?

klahnakoski commented 5 years ago

The ES ingestion had started to fall behind Wed Aug 7, I noticed on Fri Aug 9th; and reboot the ETL machines, and hoped it would catch up over the weekend. Mon Aug 12 showed no progress, so a fix was found (https://github.com/klahnakoski/ActiveData-ETL/issues/58), over the week ES ingestion proved to be slow, but I was optimistic. Fri Aug 16th the ingestion was still far behind (https://mozilla.logbot.info/activedata/20190816) so I turned down coverage ingestion so the rest of the data could catch up.

Over the weekend, ingestion improved and it is now working on Aug15. Hopefully it will be almost done by the end of today. More on #activedata

klahnakoski commented 5 years ago

@Archaeopteryx please confirm that the problem has gone away, and close this issue if so. Thank you

klahnakoski commented 5 years ago

@klahnakoski for my own notes: I think this is the cause: https://github.com/mozilla/ActiveData/issues/160

Archaeopteryx commented 5 years ago

Thanks, looks okay to me.