Refactor catalogue library

[seanprivett]

• This can take the format of a pair/group coding exercise, like when we built the django forms and services in find-moj-data
• The catalogue library needs updating to reflect current requirements
• For example:
  • References to data products should be removed
  • Check for any platforms-related hardcoding
  • Check that upsert methods exist for all currently used data types (table, database, and chart)
  • Check how we do error handling for missing data in DataHub. Does this fit nicely with displaying errors in the frontend?
  • If a domain doesn't exist, registration of an entity with datahub fails. Find a more elegant way to handle this/recheck this is the behaviour we want
  • Currently our get_table_details datahub client method doesn’t error if the urn doesn’t exist, just returns a result like the table exists but with no metadata.
  • get rid of the catalogue base class

[tom-webber]

• Handle 'Security classification' / sensitivity level / PII more deliberately
  • remove security classification for now
• Search
  • remove references to Data Product
• Structure:
  • add upsert methods to DataHubCatalogueClient for each type of entity we want to add (e.g. database, table, chart)
  • upsert_table vs. upsert_athena_table: can upsert_table be a generic method where 'platform' is passed
• GraphQL
  • add sub-structures of graphql queries as re-usable partials (e.g. Dataset details)

[murdo-moj]

• Do we want a name as well as a display name on an entity

[murdo-moj]

https://github.com/ministryofjustice/data-platform/pull/4115