search

mozilla / fxa-activity-metrics

A server for managing the Firefox Accounts metrics database and pipeline
1 stars 3 forks source link

Export sessionToken and device counts to fxa-basic-metrics #97

Open philbooth opened 6 years ago

philbooth commented 6 years ago

Currently we export a daily count for accounts and verified accounts to the fxa-basic-metrics bucket in S3 and store it in Redshift in the counts table. It would also be interesting, from a back-end dev perspective, if we could keep an eye on counts for the sessionTokens and devices tables without needing to pester ops folk every time we want to know the answer. Either by appending columns to the current CSV, or by putting them in a fresh S3 bucket if that's preferred.

Obviously super-low priority and absolutely not the end of the world if we don't do it. I just often find myself wondering about the sessionTokens table is all.

vbudhram commented 6 years ago

from mtg: Low priority, nice to have

philbooth commented 6 years ago

Does this fit in with KR 2.2 potentially? Tentatively putting it in to that milestone, but feel free to move it out if that was wrong.

rfk commented 6 years ago

@philbooth just touching this during backlog grooming, does it still feel like it should be part of this Q1 milestone?

philbooth commented 6 years ago

does it still feel like it should be part of this Q1 milestone?

It does to me! But I'm biased: it's something I want from a dev perspective and doesn't involve much work to accomplish. It seems like the kind of thing that will never get done unless it's included in a milestone.

But I'm relaxed about the prospect of it being punted if it's not valuable to the team as a whole.

philbooth commented 5 years ago

Hey @jbuck, I had this old idea for the counts table in Redshift, which we never got to prioritising. Seeing as we've been touching this stuff recently and (presumably) it's a simple change, maybe we can do it now?

To save you the effort of scrolling up: I'd like to export COUNT(*) for the sessionTokens and devices tables in the same way we already export it for accounts. Can we do that? I'll prep a PR for the import scripts if so.

Secondary question: What if we just wanted to blanket-export counts for every table in the fxa schema? Would that be a huge pain in the arse? Admittedly I don't have a particular use in mind for the other tables but it might be a handy signal to include on some dashboard and gives us devs some perspective on the current scale of things in the prod db, which is mostly opaque to us currently.

jbuck commented 5 years ago

Yeah, we can do this pretty easily. Existing script is in https://github.com/mozilla-services/cloudops-deployment/blob/master/projects/fxa/puppet/modules/fxa_admin/files/data/scripts/basic-metrics.sh

On Wed, Jan 2, 2019 at 6:04 AM Phil Booth notifications@github.com wrote:

Hey @jbuck https://github.com/jbuck, I had this old idea for the counts table in Redshift, which we never got to prioritising. Seeing as we've been touching this stuff recently and (presumably) it's a simple change, maybe we can do it now?

To save you the effort of scrolling up: I'd like to export COUNT(*) for the sessionTokens and devices tables in the same way we already export it for accounts. Can we do that? I'll prep a PR for the import scripts if so.

Secondary question: What if we just wanted to blanket-export counts for every table in the fxa schema? Would that be a huge pain in the arse? Admittedly I don't have a particular use in mind for the other tables but it might be a handy signal to include on some dashboard and gives us devs some perspective on the current scale of things in the prod db, which is mostly opaque to us currently.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/mozilla/fxa-activity-metrics/issues/97#issuecomment-450836896, or mute the thread https://github.com/notifications/unsubscribe-auth/AAjTouYylhtUrl3mJxgy_DnRElPpqxyhks5u_JJZgaJpZM4RmhZV .