davidkel / hlf

0 stars 0 forks source link

[fabric-network] Guaranteed event delivery while application is running only possible if checkpointing is used #38

Open davidkel opened 4 years ago

davidkel commented 4 years ago

As fabric-network controls the event hubs and their registration, unless checkpointing is enabled then there is a window of opportunity where loss of an event hub connection and registration of a new event hub connection 1 or more blocks could have been committed and as such a block could be missed and thus events could be missed.

If developers want to do their own checkpoint management (not using the checkpoint interface) for example they already track the block number as part of handling the event received and when the application restarts will provide that block number as part of the registration however they cannot do this while the application is up and running as they don't have control of the event hub registration on a failure (unless they somehow deregister the listener if an error is received and re-register the listener but this is getting complex and it's not clear if there could be timing consequences between the application code and the fabric-network code in facilitating this)

The issue here is : Should checkpointing be the ONLY way to provide guaranteed event delivery and if so then does allowing a user to specify startBlock (or even endBlock) value have any practical value ?

Maybe a solution is that unless a checkpointing request is explicitly made, fabric-network implements an in memory checkpoint mechanism that is used to ensure that while an application is running events are guaranteed to be delivered. Then the application can decide whether to implement a checkpoint interface or pass startBlock when registering depending on how they want to do things.

If not then it should be explicitly documented on what is required to achieve guaranteed event delivery on a practical level, both for development testing and production (ie file system checkpoint is not a practical option in most cases)