caldeirav
commented
3 months ago With the backbone of the platform now solidly on Iceberg, and the plan to introduce Unity Catalog for federated data governance, we should evaluate whether the best way to access the data would be leveraging the Unity Catalog UI:
https://github.com/unitycatalog/unitycatalog-ui
Pros:
- Integration of the UI with Iceberg data structure / metadata information
- Should be easier to ensure governance / access layer is in line with federated governance requirements
Cons:
- There is still a need for data engineers to be able to run SQL requests against Trino and test output
As a data engineer, I want to be able to browse existing catalogs and data ingested in Trino.
For this, it is suggested we install a cloud database management tool such as Cloudbeaver community (https://github.com/dbeaver/cloudbeaver) or similar. This tool has been used successfully before by OS-Climate. Ideally it should be deployed with an authentication provide in line with our management of access in Trino.