`chainHead_follow`: unpin old blocks automatically instead of stopping and unpinning all blocks.

[jsdw]

This is a followup to #100 to tweak the suggestion a bit and hopefully present it better.

Currently, the chainHead_follow subscription emits a "stop" event and ends (unpinning all currently-pinned blocks) if a block if old blocks are not unpinned by the user within a (currently unspecified) time frame.

Proposal

I'd propose that the chainhead_follow subscription emits an { "event": "unpinned", "hash": "0x123" } style event for any block that is not unpinned by the time it gets too old for the server to keep around. In other words, instead of the server emitting a "stop" event when encountering a block that it wants to unpin, emit an "unpinned" event instead for that block.

• If you prefer the current behaviour, you can simply stop the subscription as soon as one of these events is seen in addition to when any "stop" event is seen.
• The "stop" event will still be emitted in any exceptional case where the subscription cannot continue (it's expected that in these cases, there was nothing that the user could do to prevent it anyway).

Motivation

My reasoning for this is that the default/easy behaviour of chainHead_follow should ensure that new blocks are pinned for as long as possible under normal operation (eg errors or warp syncing may lead to the subscription not being able to continue, which is unavoidable). Then, the user can help to "optimise" the behaviour of the server by manually unpinning blocks that are no longer needed itself.

In my view, this:

• Ensures that the API is simple to use; chainHead_unpin can just be ignored in the simple case, and we don't need to track block ages in the client to know when to make chainHead_unpin requests.
• Ensures that newly presented blocks are pinned for as long as possible (or until the user explicitly does not care about them any more); we don't have to defensively unpin a block any sooner than necessary "just in case".
• Protects against any latency or network sisues that may prevent chainHead_unpin requests from making it to the server in time.

On the other hand, the current approach has a couple of issues:

1. We don't know how long the server is able to keep blocks pinned for, and so we have to make "unpin" requests for blocks arbitrarily earlier than might be necessary to play it safe and prevent the subscription from stopping.
2. Even if we did know how long the server would keep blocks around for, we need to be a little defensive incase of any network issues, and unpin a little sooner than the max age, and so still cannot keep blocks around for the maximum possible time.
3. We have to keep track of all of our pinned blocks and their ages in order to know when to make "unpin" requests; this is just slightly tedious book keeping that the server already has to do, so could be avoided.
4. If we mess up with either of these, or encounter latency or whatever which prevents our chainHead_unpin requests from getting through in time, everything is unpinned, and we must restart the subscription ourselves to continue, possibly missing some useful information in the interim.

Alternatives

A couple of alternatives are:

1. Specify the minimum amount of time blocks are pinned for.
   • Now, the client knows roughly when it needs to call "unpin", but different servers may be able to keep blocks around for much longer than this minimum, and so we still lose out of holding blocks for as long as might be possible.
2. Add an API call eg chainHead_maxBlockAge so that the server can tell the client how old blocks are allowed to get.
   • We solve (1), and can be more flexible now in different server impls re how long blocks are kept for, but we still have to be a little defensive, still need our book keeping on block ages etc.
   • We also add another API call and the documentation/understanding that is needed to go with this.
   • But, this might be useful for anticipating up front what will be possible and what won't be, so could also exist in parallel.

[jsdw]

Closing this; not enough support for the idea!