aberaud
commented
3 years ago Hello and thank you for the idea,
There currently are client-side (HTTP client) and server-side (DHT node) caches for 'listen' operations, but not for 'get' operations at the moment, unless a 'listen' operation is ongoing for the same key. In that case, the cache is used to answer get operations. Additionally, if values are stored by or on the DHT node itself with a 'put' operation, those values will be returned immediately in case of a 'get' or 'listen' operation, so that a DHT node can be used in the way you describe - it would act as a standalone key-value store in case of disconnection from the DHT.
So we'll consider those new features:
- Adding a new REST API query parameter for 'get' operations to limit the number of returned values. Note that the operation can already be canceled at any time by closing the connection. Impact on performance would be limited.
- Adding a DHT node cache for 'get' operation. Might be implemented by performing a fake listen operation on the key for the wanted duration of the cache (there would be no need for max-age parameter since the node would then remain synched for this key).
ghenry
At present, the REST API seems to have no way to control the number of returned values, and seems to only briefly cache values, if at all.
To increase performance, I think that a limit parameter would be very helpful, especially for the common case in which you only expect one result, and only want one.
I also think that some kind of caching functionality would be extremely useful. There are many potential use cases where a small number of popular keys could possibly be requested thousands of times. Proxy side caching (potentially controlled by a maxage parameter in the request) could reduce the bandwidth in such cases.
It would also allow for high-reliability on premises proxies. Even if a DoS were to affect the DHT as a whole, anything published via your on-premises proxy would still be visible to other local network nodes via the cache, meaning the whole system can be used in slightly more mission-critical roles.