ministryofjustice / analytical-platform

Analytical Platform • This repository is defined and managed in Terraform
https://docs.analytical-platform.service.justice.gov.uk
MIT License
8 stars 4 forks source link

📊 Bootstrap Lake Formation in Analytical Platform Compute and APDP Accounts #4461

Open julialawrence opened 1 month ago

julialawrence commented 1 month ago

User Story

In order to deliver the QuickSight MVP backed by Lake Formation-managed Athena permissions, we would like to bootstrap and configure Lake Formation in our Analytical Platform Data Production and Analytical Platform Compute accounts.

Value / Purpose

Lake Formation is a superior approach to managing access to resources registered in Glue especially in scenarios where resources are shared cross-account. Therefore, in order to begin transitioning from using IAM for access to Lake Formation, QuickSight MVP will be a test bed for the approach, with assets managed solely in LF.

Useful Contacts

@julialawrence @michaeljcollins @jamesstott

User Types

Data Engineers

Hypothesis

If we implement Lake Formation, this will reduce our and data engineering overhead in managing access as well as provide more granular permissions management approach.

Proposal

The purpose of this story is to configure Lake Formation in analytical-platform-data-production and analytical-platofrm-compute accounts to allow cross-account, cross-region sharing of assets using named resources method. Enabling sharing of tags is a stretch.

This will require implementing the following: *A service IAM role in each account with broad LakeFormation, Athena, Glue and S3 permissions to be used as service roles for the UI to manage sharing.

Additional Information

https://docs.google.com/document/d/1Xwbvc8ipI2m6nlK3et-TUcrksjhV9MmNqEVFtVbtkOQ/edit

Definition of Done

jacobwoffenden commented 2 weeks ago

Blocked by #4358