add `withHeader` option to `chainHead_unstable_follow`

[josepot]

I am proposing a small yet significant change in our API to introduce a withHeader option. This enhancement is driven by the need to simplify and optimize the way consumers interact with the chainHead_unstable_follow subscription, particularly in scenarios where understanding the topology of observed blocks is important.

Rationale:

In our current setup, there are critical instances where a consumer needs to discern the lineage of new blocks, especially those belonging to the "best blocks" chain – blocks that, while not yet finalized, have the current "best-block" as an ancestor. The critical case involves a new block that brings a runtime upgrade. Here, the consumer is required to download the latest metadata corresponding to that runtime upgrade. This ensures that any subsequent requests targeting descendants of that block are compatible with the correct version of the runtime.

Under the existing API, achieving this demands a rather cumbersome process. Consumers are forced to eagerly request headers for all observed blocks and subsequently hide from the final consumer blocks whose fork affiliation remains unidentified. This approach is particularly problematic when dealing with non-light-client JSON-RPC servers. It can lead to significant back-pressure issues and unnecessary bandwidth consumption due to the constant stream of chainHead_unstable_header requests. While light-client servers may not face such challenges due to in-memory storage of headers, the inefficiency remains a concern for other server types.

Solution:

By integrating a withHeader option as proposed in this PR, we can effectively avoid these issues. This addition would streamline the process, reducing the need for redundant data requests and processing. It not only alleviates the burden on the consumer's end by simplifying the interaction model but also enhances the overall efficiency the process, mainly in non-light-client server contexts.

I believe this change, while seemingly minor, could have a substantial positive impact. I look forward to feedback and further discussion on this proposal.

cc: @tomaka

[josepot]

I just realized that I don't need to full header as the parentBlockHash is already present on the newBlock event 😅 . Super sorry about this!