search

mozilla / fxa-activity-metrics

A server for managing the Firefox Accounts metrics database and pipeline
1 stars 3 forks source link

feat(scripts): add country and region columns to activity/flow events #123

Open philbooth opened 5 years ago

philbooth commented 5 years ago

Fixes #108.

Adds 2 extra columns, country and region, to the activty_events and flow_events tables. We need to get those 2 columns added to the CSV before we can apply this change (hence WIP), because without them the import will fail. This also means that, once applied, the scripts won't be able to re-import old data from before the columns existed.

@jbuck, based on the discussion in https://github.com/mozilla/fxa-activity-metrics/issues/108#issuecomment-450213718, do you think stackdriver logging will happen soon enough for us to just add columns to the CSV that way, or do you think we'll need to update the Heka filter as an initial step? I suspect it's the latter, but worth asking.

And if we do need to update the Heka filter, do you know who we should ask to help us? I used to know how to do this, but I can't remember what was involved now, or even where to find the code.

jbuck commented 5 years ago

Yes, I think we'll have Stackdriver logging in place in early January

philbooth commented 5 years ago

Yes, I think we'll have Stackdriver logging in place in early January

This is brilliant news!

jbuck commented 5 years ago

@philbooth will I need to run a manual DML change to add these two columns to the appropriate tables?

jbuck commented 5 years ago

Also, can we merge this now? or will the additional columns not being present in the current data set cause problems?

philbooth commented 5 years ago

will I need to run a manual DML change to add these two columns to the appropriate tables?

Yep, that's what I was thinking.

Also, can we merge this now? or will the additional columns not being present in the current data set cause problems?

I'm pretty sure the missing columns will cause it to fail. ISTR trying that before, a long time ago.