Measure p90 latency and failure rate of HUD PR and commit pages

[huydhn]

Pitch

After all recent Rockset and GitHub outages, I think we should start tracking some KPI for important developer facing HUD pages like PR and commit pages. Here are some observations:

• It takes a noticeable amount of time to load these pages fully, from few seconds to half a minute. Having the metric helps us balance between adding more features and performance, especially if we want to push for HUD adoption among developers (everyone loves pages that load faster)

• These page returns error sometime when its upstream dependencies fail, probably GitHub API but I'm not entirely sure. So it makes sense to start thinking about availability

Solutions

TBD. May be vercel already has these metrics somewhere that we can just tap in

cc @janeyx99 @ZainRizvi @pytorch/pytorch-dev-infra

[ZainRizvi]

We def need to get better about tracking metrics on our tools in general, at the least storing them somewhere to help with future debugging.

For prioritization: I'd suggest tracking usage of the tools (like who/how often people even click on HUD links) is the top priority, and then evaluating the quality of those experiences comes in second

[huydhn]

True, we have that other task to track tool usage mapped out at https://github.com/pytorch/test-infra/issues/530. So we can address it before this one