search

dat-ecosystem-archive / dat-dns

Issue DNS lookups for Dat archives using HTTPS requests to the target host. [ DEPRECATED - see https://github.com/martinheidegger/hyper-dns for similar functionality. More info on active projects and modules at https://dat-ecosystem.org/ ]
57 stars 13 forks source link

Reads from persistent cache are not retained in memory cache #22

Open sammacbeth opened 5 years ago

sammacbeth commented 5 years ago

When providing a persistentCache option, the cache is only read if there is a memory cache miss, and the lookup fails. In this case, the result from the persistent cache is not saved to the memory cache. When this happens, it is likely to be a repeated failure (e.g. device is offline), so the persistentCache will be repeatedly hit for every dns request. As the cache is persistent, it is likely to be much slower to retrieve results. Maybe it makes sense for the persistentCache result to be saved in the memory cache?

Perhaps it would also make sense to consult the persistent cache directly after a cache miss, to allow lookups without hitting the network after an app restart, for example. This would likely require the cache to also return a ttl though.

pfrazee commented 5 years ago

+1 on both suggestions: saving the p-cache read to the m-cache, and storing the ttl to the p-cache to allow for cached reads after restart. @sammacbeth do you have time to PR those updates?

sammacbeth commented 5 years ago

Sure, I'll try have a look this week.

I'm wondering how this could be added without causing a breaking change. At the moment read should return a string or throw an error. This gives no space to add an extra ttl value. One options would be to just assume a short ttl (say a few minutes) for the m-cache. This would prevent the p-cache getting hammered, but not require a change to it's API shape.