Detail view quite slow when viewing larger intervals of data

[wlach]

The detail view seems to work pretty well for smaller intervals (24, 48 hours) but seems to get kind of slow when viewing larger ones (e.g. 7 days). I think most of this is metricsgraphics/d3 being slow with this quantity of data -- not sure how easy that is to fix. Another solution might be to only display data on an hourly interval for older data (probably taking the median value). That will still show changes over the course of each day without slowing the ui to a crawl.

We could probably do this in the getTransformedSeriesList method in the detail view. Likely if we do this sort of filtering, we should just ignore the results of the percentile filter:

https://github.com/mozilla/missioncontrol/blob/a1d0044cbf1d0d113ccfc5756379680584af3a8c/frontend/ui/detailview.jsx#L21

[akmittal]

Backend returns 72 KB for 24 hours and 1.2 MB file for 14 days data which is huge. Would it be possible to return lesser(interval) data from services?

[wlach]

We could do it at that level, but as I said, the slowness is mostly on the client. Downloading/fetching the data is relatively fast. My preference would be to leave things as-is there, in case some other consumer of the API wanted a complete set of data.

[wlach]

I think I've changed my mind on the above: let's still use the percentile filter, it should still be useful (since the reduced data will still be affected by the calculations).

I think it makes sense to do this in the same place I mentioned above though. Here's one possible implementation strategy:

• Filter the data for the most recent 48 hours, take this verbatim
• For data older than 48 hours, chunk it in hourly intervals (so intervals of 12, can use lodash's chunk operator here: https://lodash.com/docs/4.17.4#chunk), then take the median of each chunk (using aggregatejs's median function: https://github.com/yefremov/aggregatejs) using map
• Concatenate the two together for each series

Hopefully that makes sense?

[wlach]

In the end I think just clamping on hourly / daily intervals makes more sense. Have a PR brewing to that effect.

[wlach]

Here's what hourly agagregates look like for 14 days worth of main crashes: