mitodl / ol-data-platform

Pipeline definitions for managing data flows to power analytics at MIT Open Learning
BSD 3-Clause "New" or "Revised" License
36 stars 6 forks source link

Grant reverse_etl role access to staging and intermediate schemas #1204

Closed quazi-h closed 1 month ago

quazi-h commented 1 month ago

What are the relevant tickets?

Not sure if this is the correct ticket for this particular task but it is related: https://github.com/mitodl/hq/issues/4621

Description (What does it do?)

We have an "OL Production Marts" source set up in HighTouch which was initially configured to use _ol_warehouse_productionmart as the default schema and utilizes the "reverse_etl" role to access our data lake. This prevented us from being able to load staging and intermediate dbt models in HighTouch. When troubleshooting and testing a new sync in HighTouch, I manually granted the "reverse_etl" role access to our production staging and intermediate schemas using the "Roles and Privileges" settings under the Access Control interface in Starburst. This was confirmed as a fix after I confirmed that HighTouch was able to access the _ol_warehouse_productionintermediate and _ol_warehouse_productionstaging models after this change was made.

This PR updates the grant privileges for the reverse_etl role to correctly reflect the configuration we have set up in Starburst/Trino. This ensures that the correct access is defined in our codebase and will ensure that HighTouch does not lose access to those data models if our current Trino access control configuration gets clobbered.