Add webhooks

[qdequele]

Pull Request

Related issue

Fixes #59, #5

What does this PR do?

The PR adds the capacity to the scraper to send updates to a webhook. This takes three new env vars:

• WEBHOOK_URL that is necessary to use the webhook.
• WEBHOOK_TOKEN is optional but needed in case of a protected webhook URL.
• WEBHOOK_INTERVAL is optional; the default value is 5000ms.

This PR also add a /webhook that logs the webhook bodies for test purpose.

The Webhook handler works as a singleton. Here is some example of webhook bodies.

Started:

{
  status: 'started',
  meilisearch_url: 'https://xxx.lon.meilisearch.io',
  meilisearch_index_uid: 'meilisearch_docs',
  date: '2023-07-23T10:05:58.768Z'
}

Active:

{
  status: 'active',
  nb_page_crawled: 8,
  nb_page_indexed: 6,
  nb_documents_sent: 62,
  meilisearch_url: 'https://ms-c231a2ea8724-106.lon.meilisearch.io',
  meilisearch_index_uid: 'meilisearch_docs',
  date: '2023-07-23T10:06:04.335Z'
}

Completed:

{
  status: 'completed',
  meilisearch_url: 'https://xxx.lon.meilisearch.io',
  meilisearch_index_uid: 'meilisearch_docs',
  date: '2023-07-23T09:56:10.951Z'
}