Performance DB improvements - Field names

[Ben-Hodgkiss]

Overview

Data Management have requested the following change to the Performance DB.

• Some field names are inconsistent between tables, e.g.
  • endpoint_dataset_summary table only has the dataset field, reporting_historic_endpoints only has the pipeline field, while endpoint_dataset_issue_type_summary has collection, pipeline and dataset. Would help to have either pipeline or dataset consistently across all tables to help with reporting summaries by dataset.
  • Organisation name is name in reporting_historic_endpoints, andorganisation_name in endpoint_dataset_issue_type_summary.

It would also be great to improve the documentation for the new tables. We’ve created a new page in the technical documentation for this to live, it just needs to be populated with descriptions of tables and fields.

Tech Approach

Consider duplicating fields for now rather than replacing - we know that these fields are being used heavily by the Providers Team and Config Manager - we need to be careful when deploying to avoid any further issues. We would rather have duplicate fields (with different field names) than do a straight rename and cause errors.

Acceptance Criteria

• Field names have been updated to the above.
• When deployed, there are no new issues experienced by the Providers services or Config Manager from renaming fields.
• Documentation has been updated for the new tables.
• A handover call (or open message in Planning Data Team) between Infrastructure, Providers and Data Management has taken place to explain the changes.
• Make a separate ticket to remove the duplicate column after all checks have been done.

Checklist

• [x] Add 'dataset', 'collection' and 'pipeline' column in reporting_historic_endpoints, endpoint_dataset_summary, endpoint_dataset_issue_type_summary table
• [x] Make organisation name consistent in all the tables (organisation_name)
• [x] Update new page in the technical documentation

[Ben-Hodgkiss]

Jira ticket here: https://mhclgdigital.atlassian.net/browse/DATA-892?atlOrigin=eyJpIjoiMDVkM2VkNTllODI5NDNiOGI0OTE2MmQyMjVkNDZlODAiLCJwIjoiaiJ9

[eveleighoj]

• endpoint_dataset_summary table only has the dataset field, reporting_historic_endpoints only has the pipeline field, while endpoint_dataset_issue_type_summary has collection, pipeline and dataset. Would help to have either pipeline or dataset consistently across all tables to help with reporting summaries by dataset.

let's ensure that all tables have the dataset field so let's add dataset to reporting_historic_endpoints so that they all have dataset

• Organisation name is name in reporting_historic_endpoints, and organisation_name in endpoint_dataset_issue_type_summary.

add organisation_name to reporting_historic_endpoint remember to leave name in so no brakages