search

ipfs / boxo

A set of reference libraries for building IPFS applications and implementations in Go.
https://github.com/ipfs/boxo#readme
Other
210 stars 91 forks source link

Gateway: ensure consistent caching of DNS records #480

Open hsanjuan opened 1 year ago

hsanjuan commented 1 year ago

The gateway code uses https://github.com/multiformats/go-multiaddr-dns which by default would use net.DefaultResolver and not cache anything. It also registers two default resolvers for .eth and .crypto. In this case using DoH urls for which it uses https://github.com/libp2p/go-doh-resolver, which does include caching of results.

This means some dns results are going to be cached while others depend on what DNS resolver Go is actually using (which depends on CGO iirc). When not using CGO, Go would use its own resolver and not cache anything. When using CGO, go would use the system's resolver which may or not cache stuff depending what is on the system.

Is my understanding correct @lidel ?

Ideally we should be caching all results. This has been used before in the ecosystem: https://github.com/rs/dnscache

lidel commented 1 year ago

Yes, we seem to lack cache for the default DNS resolver from OS. Having universal cache in boxo/gateway makes sense.

cc @hacdias - you have been looking at this in context of TTL, thoughts where would be best place to wire it up?

hacdias commented 1 year ago

I'm working on reworking the namesys package here: #459. This will allow us to bubble up TTLs to the gateway and actually use them in requests. I don't mind adding support for DNS TTLs - but perhaps in a separate PR.

I also want to mention that DNS caching has been talked here before: https://github.com/multiformats/go-multiaddr-dns/issues/28

hsanjuan commented 1 year ago

TTL bubbling is a bit different from caching. Just saying there should be a layer (perhaps above Resolver) that does the caching consistently for everything that is resolved (based on TTL, why not).

hacdias commented 1 year ago

Yes, #459 also includes caching, but just for IPNS. If the DNSResolver were to return a TTL, namesys (which uses both IPNSResolver and DNSResolver) would cache resolved dns lookups.