search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.97k stars 3.79k forks source link

roachprod: investigate concurrent calls to `gcloud` DNS safety #122180

Closed herkolategan closed 1 week ago

herkolategan commented 5 months ago

At the time of writing we noticed that multiple API calls to update the same records, with the same info, were originating from the gcloud CLI calls to the DNS API server. These API calls to the HTTP endpoint was made by gcloud. And only one call was made from roachprod. This could be "normal" for gcloud, or it could be some interference when multiple updates are being made to the same zone, on the same VM.

This issue is to ensure it's safe to do concurrent calls on the same VM for the same DNS zone. It's already established that the gcloud DNS transactions commands do not allow overlapping transactions even if it is for different records. If it's the same zone gcloud fails the command.

Jira issue: CRDB-37747

blathers-crl[bot] commented 5 months ago

cc @cockroachdb/test-eng

herkolategan commented 1 week ago

Have not seen this issue in a while after introducing thread safety for the DNS implementation, and after a bug fix on Google DNS services from a google support ticket.