Deploy Browsertrix Cloud

[anjackson]

We have been deploying Browsertrix Cloud under K3S. Making this work requires some changes to the base Browsertrix Cloud service so we can deploy locally.

• Must be possible to specify a suitable User Agent string (includings spaces).
  • See https://github.com/webrecorder/browsertrix-crawler/issues/307 and around
• Must be possible to use a specific set of external DNS name servers.
  • See https://github.com/webrecorder/browsertrix-cloud/issues/1137
  • Nevermind, setting the DNS config at the K3s level is fine.
• Must be possible to send emails using our internal mail server (Port 25, not TLS, no auth).
  • See https://github.com/webrecorder/browsertrix-cloud/issues/1136
• If possible, support deployment via Traefik (or re-install K3S using ingress-nginx instead).
  • See https://github.com/webrecorder/browsertrix-cloud/issues/1135
• [ ] Make sure all important configuration has been checked, e.g. WACZ file naming
• [ ] Integrate deployment setup+overrides into this repo.

[anjackson]

Still awaiting a 1.7.x official release that includes the SMTP changes we need, but I've restarted the cluster and run a crawl to check the User-Agent and it's all looking good.