search

awslabs / aws-orbit-workbench

A Data Platform built for AWS, powered by Kubernetes.
https://awslabs.github.io/aws-orbit-workbench/
Apache License 2.0
127 stars 26 forks source link

Fix LakeFormation lake creator notebook. #705

Closed stthoom closed 3 years ago

stthoom commented 3 years ago

Fresh deployment of fnd->env-teams works well. As part of it we are creating a cms_secured_db glue database and registering the secured bucket path with LakeFormation.

Second iteration -> When deploying teams again, as part of destroy, we drop the previously lake formation registered bucket and create new one.

As part of regression lake-creator notebook, we try to drop the cms_secured_db and create new one.

Need to figure out deleting cms_secured_db -> old(unavailable) secured bucket path forcibly in lake-creator regression notebook. From AWS console UI, still see orbit-dev-env-lake-creator-role has all permissions - All, Alter, Create table, Describe, Drop - over the database

stthoom commented 3 years ago

Debug - Check if making lake-creator as lakeformation admin can forcibly delete the cms_secured_db pointing to dropped s3 bucket.

stthoom commented 3 years ago

Approach - Using Orbit script runner plugin destroy hook, delete the LakeFormation notebook specific resources( cms_secured_db, etc).