search

snarfed / bridgy-fed

🌉 A bridge between decentralized social network protocols
https://fed.brid.gy
Creative Commons Zero v1.0 Universal
667 stars 34 forks source link

Scale ATProto handle resolution #744

Open snarfed opened 11 months ago

snarfed commented 11 months ago

We need to serve ATProto handle resolution for all users bridged into ATProto (background: #381). They support both DNS and HTTPS methods, but our ATProto handles are multi-level, eg @user@mastodon.social becomes user.mastodon.social.ap.brid.gy, and you can't make multi-level wildcard SSL certs, so HTTPS won't work, so DNS it is.

We're currently using Google Cloud DNS. It serves the entire brid.gy DNS zone, and we create _atproto records for handle resolution manually. The catch is that GCP DNS has a hard limit of 10k records per zone, which we'll likely outgrow. Grr.

We could make a zone per sub-subdomain, eg per fediverse instance, so mastodon.social.ap.brid.gy would become its own zone, but GCP DNS also has a hard limit of 10k zones total.

So, we eventually need to switch to a different programmatic DNS service or run our own DNS server. Whee.

snarfed commented 11 months ago

Provider lists: https://en.wikipedia.org/wiki/List_of_managed_DNS_providers https://www.g2.com/categories/managed-dns-providers https://serverauth.com/posts/comparing-dns-hosting-providers

snarfed commented 10 months ago

Upcoming option: https://dns.kitchen/ , all you can eat zones/records, $5/mo.

snarfed commented 9 months ago

@neatnik mentioned https://desec.io/ too.

snarfed commented 9 months ago

Also https://dns.he.net/, https://www.nova53.net/, https://ns-global.zone/ .

snarfed commented 9 months ago

Maybe obsoleted by #830 🤞

snarfed commented 7 months ago

...nope, turns out #830 probably won't work after all, so this issue definitely still applies.

snarfed commented 7 months ago

The 10k limit in Google Cloud DNS is a quota, rrsetsPerManagedZone, that I can request an increase for. May try that.

snarfed commented 6 months ago

Tried requesting a quota bump. 🤞

snarfed commented 6 months ago

In unrelated very good news, @bnewbold built us a whole new microservice to solve this! https://github.com/bluesky-social/atproto/discussions/1697#discussioncomment-9308726 . Extremely generous of him. Thank you Bryan!!!

snarfed commented 6 months ago

More good news, I asked for a GCP DNS quota bump to 50k and got it. Woo!

snarfed commented 3 months ago

We're currently at 6500 DNS records, with the limit still at 50k. We'll (hopefully) still need to figure this out eventually, but it seems like we have plenty of time.

snarfed commented 4 days ago

Count is 28k now! Out of quota of 50k. 😳 I think a decent number of those are disabled, many but not all from Flipboard.

To do: