Source of truth is unclear

[sevanspowell]

The design of this project stores the metadata in two places:

• In the metadata storage layer, in this case a Postgres database, which is read from by the metadata-server and written to by the metadata-webhook
• The GitHub repository (usually accessed via the web UI)

There is no single source of truth and the project provides no way to ensure the two sources are not out-of-sync with each other.

[sevanspowell]

The only way data is moved from the GitHub repository to the database is via the webhook. Unfortunately the webhook is prone to fail: flaky internet, the GitHub API not being available, etc. will cause the webhook to fail. If the the webhook doesn't run, the database gets out of sync with the GitHub repository and this has caused issues, see: https://github.com/cardano-foundation/cardano-token-registry/issues/61 and https://jira.iohk.io/browse/ADP-892 (IOHK only - sorry!).

Having two sources of truth is not ideal, but until we arrive at a better design, I propose some interim solutions:

• 1. implement an update script, that runs multiple times a day, to sync the database with the GitHub repository
     • This way if the webhook fails, the update script can pick up the slack
• 2. Remove the database entirely, re-jig the metadata server to serve files from a local clone of the metadata repository
     • This will significantly minimize potential sources of error when syncing the database and the GitHub repository.
     • Make syncing the metadata-server as simple as git pull
     • Allow operators running their own metadata-server to easily use alternative registries ala https://github.com/input-output-hk/offchain-metadata-tools/issues/2

Provided it is performant, solution 2 is the lowest effort, highest assurance option. Any code I write is going to be significantly more prone to failure than a git pull.