alrevuelta
commented
11 months ago Consider the moment when a Store server can no longer store the whole history. What choice does it have?
In nwaku this is curently solved with the retention policy https://github.com/waku-org/nwaku/blob/master/apps/wakunode2/external_config.nim#L230-L233 (could be time based, amount of msg based or size in Gb based).
imho we should target 30 days, being that the default value. As per https://github.com/waku-org/research/issues/31 if we set the max traffic per shard to 10 Mbps, assuming a D=6 thats 10/6=1.6Mbps or 0.2 MBytesps of messages to store. Of course assuming a 100% network utilization. That is (worst case):
- 750 MB/hour
- 18 Gb/day.
- 540 Gb/month.
But well, with these numbers perhaps its more reasonable to say 1 week (126 GB), which is still quite high. Or we may have to revisit the max bandwidth set in https://github.com/waku-org/research/issues/31.
Store servers are implicitly supposed to store all messages forever. How sustainable is this?
On the one hand, infinite linear state growth is not necessarily unacceptable (Bitcoin operates under this model just fine). Bitcoin, however, has a limited and predictable rate of growth (the maximum of 4 MB / 10 min on average). A message-centered network can expect more traffic.
How sustainable is this model in our case?
Consider the moment when a Store server can no longer store the whole history. What choice does it have?
The issue is that expectation of a client are violated in the latter case. A client expects each server to be able to serve any past message, when in reality it is not the case.
One way to address this could be #37 . Another (complementary) idea may be to introduce a limit on the age of historic messages that servers store (e.g., 1 month). Clients then can be certain that they can query messages from the past month from any server, and for older messages they either need to negotiate, or expect some rejected queries. From the server's perspective, meanwhile, not serving messages older then default does not constitute misbehavior and does not cause punishment (if such punishment is implemented as part on incentivization scheme).