CloudFlare Rate Limiting

[augustuswm]

sync-shorturls (and by extension other functions like sync-other) has started running into CloudFlare rate limiting:

More than 1200 requests per 300 seconds reached. Please wait and consider throttling your request speed

Currently every record performs a list request, and as such can easily exhaust the limit. A few options:

• Add forced delay to ensure we are below the limit
• Fetch and cache all records up front during the sync process
• Use DNS resolution to verify that a name resolves as expected instead of using CloudFlare API (this does not provide the same guarantees as what the sync currently does)

[augustuswm]

Each call to ensure_record seems to make a call to get_zone_identifier which likely requests the same data repeatedly and can be cached.