✨ Enable a create-a-derived-table github action integration for the EM data store

[pricemg]

Describe the feature request.

Enable the ability to run create-a-derived-table jobs using the data-platform self-hosted runners where the runner is authenticated with permissions to access resources in the EM data store MP account.

Describe the context.

Electronic monitoring data contains sensitive information on individuals (known as "specials") that the EM SLT have deemed should not be accessible outside of preapproved persons. However, to identify this data the data must be linked together (via create-a-derived-table transformations). It is only after doing this the specials data can be redacted, which the EM SLT do not want to happen in the AP.

Building on the work Digital Prisons Reporting have already had done, we have identified an agreed solution that would involve keeping EM data in the EM data store modernisation platform account and running create-a-derived-table in that account and doing all the relevant linking and redacting of data. Once done, we will use Lake Formation to expose the redacted data into the AP.

Value / Purpose

This will allow better data management and access control of sensitive electronic monitoring data.

User Types

EM create-a-derived-table users

[simon-pope]

To be presented at Request Forum 14/10

[simon-pope]

Request Forum: Required for 14th November, ask understood. Will take to Amigos and refinement w/c 21/10. Planning to take to sprint 25.

Create new Epic or User Stories as part of Amigos to be brought in to sprint.

[simon-pope]

@julialawrence to create a story based on the requirements to be taken to refinement

[jacobwoffenden]

@julialawrence @simon-pope,

@matt-heery has already started this work and I've approved it based on the existing pattern, not sure it needed a separate anyway 🤷

[jacobwoffenden]

Deployment of test runner blocked by CNI tagging change