Flow imports stopped working

[philbooth]

Just raising this for visibility, but it looks like the flow imports stopped working just before Christmas:

fxa=# select max(begin_time)::date from flow_metadata;
    max
------------
 2017-12-20
(1 row)

I'm guesssing it's related to bug 1426332, but I've disabled the cron job for now and am running a manual import instead so it can catch back up while I can keep an eye on any errors.

[philbooth]

I also clobbered a couple of days that appeared to have partially-imported data (activity_events was populated but e.g. flow_metadata wasn't).

[philbooth]

activity_events is done:

fxa=# select max(timestamp)::date from activity_events;
    max
------------
 2018-01-07
(1 row)

Imports for the daily summary tables and the flow events are in progress. Email and count imports will begin in turn after the flow stuff finishes.

[philbooth]

daily_activity_per_device and daily_multi_device_users are done:

fxa=# select max(day) from daily_activity_per_device;
    max
------------
 2018-01-07
(1 row)

fxa=# select max(day) from daily_multi_device_users;
    max
------------
 2018-01-07
(1 row)

[philbooth]

The flow import is going really slowly fwiw, it's all those extra performance events that we're still emitting I guess. It will be tomorrow at the earliest for that data.

[philbooth]

The remaining imports have finished and I've reinstated the cron job, closing this.