search

mozilla / fxa-activity-metrics

A server for managing the Firefox Accounts metrics database and pipeline
1 stars 3 forks source link

Scheduled Queries are Not Running #85

Closed davismtl closed 7 years ago

davismtl commented 7 years ago

Queries like this one haven't run in 19 days but are scheduled to run daily: https://sql.telemetry.mozilla.org/queries/1958#3631

I've had to go in a refresh many of them manually but that's no good.

philbooth commented 7 years ago

Fwiw, honestly I wish there were no scheduled queries at all. Periodically they've been the source of problems for us when they collide and then lock up resources, preventing other queries from running. The problem is that nobody goes back to turn off scheduled queries after they stop being useful, so we're left with a growing accretion that becomes worse with time, like a cancer or Kessler syndrome.

Most queries are quick to execute now, so they only take a few minutes to refresh. And unscheduled queries are self-maintaining, in that when they stop being interesting to anyone they also stop consuming resources. That means the remaining interesting queries can run even faster and the wait time for refreshes gets shorter, so we all end up better off.

None of my queries are scheduled, I have a pool of about 20 split over 5 dashboards that I refresh regularly, with maybe another 20 that aren't on dashboards that I refresh less often. If I added those 20 or 40 to be scheduled it would make everyone worse off, but I don't find the manual refresh process a burden.

davismtl commented 7 years ago

Well, for the most part, the only ones I know that are scheduled are those in our dashboards.

The problem really comes when I want to look at our product dashboard. https://sql.telemetry.mozilla.org/dashboard/fxa-overview?p_days=60&p_start_date=2017-03-01

I don't want to have to refresh and wait for each query one by one. An analysis that should take 5-15 minutes will now take a couple of hours.

This problem won't exist by the end of the quarter once we get Amplitude up and running but I still think we should be able to easily monitor our product metrics until then.

vladikoff commented 7 years ago

Should be fixeD!