search

mozilla / fxa-activity-metrics

A server for managing the Firefox Accounts metrics database and pipeline
1 stars 3 forks source link

Stopped receiving send tab metrics #125

Closed davismtl closed 5 years ago

davismtl commented 5 years ago

See received tabs: https://analytics.amplitude.com/mozilla-corp/chart/new/voy6ilt

Sent tabs: https://analytics.amplitude.com/mozilla-corp/chart/new/c8xzmx2

davismtl commented 5 years ago

CC @irrationalagent

irrationalagent commented 5 years ago

Looks like this was around the same time we made the changes to move over to the new redshift server, @philbooth any ideas?

philbooth commented 5 years ago

@rfk, I think this was the import job that I couldn't remember anything about when we talked on Vidyo before Christmas. I can't remember what we did then (and can't SSH onto fxa-redshift-helper from my new machine yet), did we remove it from the crontab?

@davismtl, we do have a plan for moving this over to be managed by ops, which is kind-of-in-progress in https://github.com/mozilla/fxa-amplitude-send/pull/79. It should be straightforward to make sure we pick up the missing data when we get that running, so it fills in the blanks. Sorry about it in the meantime though.

@jbuck, have you got time tomorrow to chat about getting https://github.com/mozilla/fxa-amplitude-send/pull/79 tested/merged/deployed on fxa-admin?

jbuck commented 5 years ago

@philbooth and I talked today about this, and I'll try out the new PR this week

ryanfeeley commented 5 years ago

@philbooth send tab has a huge impact on retention (double it for sync users alone). we're working on a CFR for send tab so this is pretty important.

philbooth commented 5 years ago

Understood @ryanfeeley, please forgive my tardiness! I'm picking it up again right now.

ryanfeeley commented 5 years ago

No worries, I had the numbers memorized as they were so thrilling to me.

jbuck commented 5 years ago

@ryanfeeley my bad, data has been imported into Amplitude now. I haven't got it scripted yet, still need to figure out the best way to do that

@philbooth how do you handle 429's for the sync dataset? I'm getting one with: node bin/sync.js events s3://telemetry-parquet/sync_events/v1/submission_date_s3=20190116/part-00000-b4087b91-5901-4f53-9652-abaca9cb0665-c000.snappy.parquet

philbooth commented 5 years ago

how do you handle 429's for the sync dataset?

I didn't have to so far because of how I was running it before. I'll have a play today and add some backoff logic.

philbooth commented 5 years ago

I'll have a play today and add some backoff logic.

Ref: #81

philbooth commented 5 years ago

We can close this now I think.