search

ministryofjustice / analytical-platform

Analytical Platform • This repository is defined and managed in Terraform
https://docs.analytical-platform.service.justice.gov.uk
MIT License
9 stars 4 forks source link

✨ Deploy DataHub into DPAT Development Cluster #2454

Closed julialawrence closed 9 months ago

julialawrence commented 9 months ago

User Story

As a member of the Data Labs team, I would like to experiment with the features of DataHub data catalogue.

Value / Purpose

Members of the data labs team are exploring potential alternatives to Open Metadata, which has issues around how it maps catalogue entities to cloud entities creating confusion

Useful Contacts

@julialawrence

User Types

Data Platform Labs

Proposal

Use Acryl helm charts to deploy, via terraform, a close-to-productionised version of DataHub in our Modernisation Platform Apps and Tools cluster, alongside with OpenMetadata - same cluster.

In order to facilitate load testing, a separate RDS postgres instance should also be created, instead of using a cluster-hosted one.

Authentication should be managed via AzureAD.

Additional Information

Chart code here: https://github.com/acryldata/datahub-helm Instructions here: https://datahubproject.io/docs/deploy/kubernetes/

The deployment process isn't straight-forward as there are quite a few configuration options and prerequisites to be deployed as well.

Definition of Done

julialawrence commented 9 months ago

Noi further work will be carried out until the productionisation is agreed.