Recreate search.page from Publishing API data

alphagov / govuk-knowledge-graph-gcp

GOV.UK content data and cloud infrastructure for the GovSearch app.

https://docs.data-community.publishing.service.gov.uk/tools/govgraph/

MIT License

8 stars 1 forks source link

Recreate search.page from Publishing API data #643

Closed nacnudus closed 4 months ago

nacnudus commented 4 months ago

The GovSearch app runs of the search.page table, which is currently derived mainly from the Content API data. This pull request recreates the same table, but from the Publishing API data, so that eventually we can abandon the Content API.

Some intermediate tables are derived, and persisted in the 'public' dataset. See individual commits for details.

The incremental updates are also fixed, since they turned out to be unreliable.