[Spike] Transfer a dashboard from job-server to Grafana

[iaindillingham]

job-server has four dashboards that could be transferred to Grafana[^1] (i.e. https://dashboards.opensafely.org/). Following a quick poll,^2 it looks like they are used as follows:

• Copiloting. Brian and Millie. Clinical Informatics.
• Private Repos. Nick.
• Projects. Liam and Juliet. Information Governance.
• Repos with Multiple Projects. Liam and Juliet. Information Governance.

For this spike, we should choose a dashboard to transfer to Grafana, and work with its users to establish that they can access Grafana and can use the dashboard effectively. We should time-box the development work to one or two days, with a short window for working with users.

Following this spike, if we choose to transfer all four dashboards, then we should write an ADR.

[^1]: The "Copiloting" dashboard in job-server isn't the same as the "Copilot dashboard" in Grafana.

[Jongmassey]

Copiloting dashboard

[iaindillingham]

Thanks for this, Jon. Couple of questions:

• Is it possible to replicate column-level sorting in the job-server dashboard?
• Similarly, is it possible to replicate column-level filtering? (Variables look like they would be useful. See the Product dashboard, too.)

[Jongmassey]

Is it possible to replicate column-level sorting in the job-server dashboard?

I find that clicking on the column header in the panel sorts it - is that not working for you?

is it possible to replicate column-level filtering?

Yes - filtering on org and project now added :)

[iaindillingham]

Ah, I see! The arrow icon appears on the sorted column, not all columns. Thanks for pointing that out. And thanks for adding filters 👍🏻

[Jongmassey]

Implementation notes from this spike on some rough edges encountered:

• the setting for enabling value filters for a Table isn't the most obvious - I've now turned this on and probably negates the need for the Copilot/Project filters (Org might be more complex on account of multiple orgs for a project being concatenated into a single field)
• The "Github opensafely" data source includes only private repos
  • therefore to infer private repos, this panel requires pulling in the list of repos from the Job Server database project table and those that are missing from the github data source are therefore private
• It's hard to resolve cardinality mismatched data in a Mixed data source
  • There is a one-to-many relationship between projects and repositories
  • When using a Mixed data source, one can only do simple single-column joins where the join key column is named identically on both sides.
  • It's seemingly not possible to do multiple sequential joins (e.g. to join tables A & B, perform some aggregation and then join this result to C)
  • It's not possible to reference an existing dashboard (e.g. where one might have already done the aggregation and formatting necessary to resolve the cardinality mismatch) in a mixed data source.

This is why the "private repos" information is in a separate panel, not inlined into a field within the main panel.

[Jongmassey]

Copilot slack thread discussing this spike

[Jongmassey]

Slack reminder set for 6 months for #team-rex to follow up on usage of this.