ministryofjustice / cloud-platform

Documentation on the MoJ cloud platform
MIT License
87 stars 45 forks source link

external-dns upgrade: constant UPSERT issues #6434

Open sj-williams opened 1 week ago

sj-williams commented 1 week ago

Background

We recently tried to upgrade to latest release of external-dns, but this resulted in R53 records taking too long to be created/updated/deleted (20mins + per operation).

We saw in the external-dns logs that the service was continually trying to action UPSERTs on all R53 records managed by external-dns.

Recreate issue in test cluster, attempt to resolve the problem