ministryofjustice / data-catalogue

Data catalogue • This repository is defined and managed in Terraform
MIT License
2 stars 0 forks source link

Scheduled metadata ingestions managed by DataHub #18

Open tom-webber opened 6 months ago

tom-webber commented 6 months ago

As of ministryofjustice/data-catalogue-metadata#6, the DataHub instances are able to fetch metadata from the production glue, athena, and dbt sources.

We should set up any ingested metadata to run on a schedule to collect fresh metadata.

Context:

Currently there's a failing ingestion ('CaDeT latest') in dev, but I've not delved into what's causing the failure