[Feature Request] Allow plugins to add tasks to chain maintenance

[vikramrajkumar]

From @nathanhourt on May 13, 2016 17:21

Plugins may add various objects to the database, which allows 3rd party code to benefit from the infrastructure the blockchain uses to maintain its internal state. One of the tools the database uses to keep its state consistent is the maintenance interval, a time during which slow calculations can be performed, or index-wide sweeps to check for and execute pending tasks can be performed.

Currently, there is no way (that I know of) for a plugin to register a routine to be run during chain maintenance. This would be a useful feature for plugins, as they also may have some pending tasks or slow calculations which can be deferred to a maintenance interval.

I propose the addition of a method on the chain::database which allows plugins to register callable objects, which take as a parameter a non-const reference to the database, to be called during maintenance intervals. These objects would need to be registered during database initialization.

A different mode of operation could involve one-off tasks which can be deferred until maintenance. In this mode, the callable objects would be registered during normal database operations, executed at the maintenance interval, and then destroyed. Note that in this mode of operation, the callable objects would have to be serializable so that they could be saved to disk and restored on database shutdown and startup. This mode of operation, although more complex to implement due to the added requirements of serializability and persistence, may be more efficient as it is assumed that the tasks would only be registered if it is known that a maintenance step will be required, and the dirty objects (the ones needing updating) may be known as state of the callable object (thus avoiding the need to sweep an entire index looking for dirty objects). In this way, plugin maintenance routines would not need to be run at all unless they were known to be necessary, and index-wide sweeps could be avoided.

Copied from original issue: cryptonomex/graphene#650

[vikramrajkumar]

From @nathanhourt on May 13, 2016 17:28

For the second mode of operation, it's possible that the plugin may process a pending task during normal operations before the maintenance interval occurs. In this case, the pending task could be canceled.

To meet this use case as well as the nominal case described above, perhaps a better design would be that plugins can retrieve a serializable object with promise semantics (i.e. a promise for a non-const reference to the database during a maintenance interval). If the promise is destroyed, the task is canceled; otherwise, the promise will resolve with the reference and a handler can run (with whatever state it needs) performing the necessary tasks.

The database infrastructure would need to persist whatever its handle for the promise is, and the plugin code would likely store the promise in a database object where it would be persisted as usual.

[vikramrajkumar]

From @nathanhourt on May 13, 2016 17:40

After more thought about this, if the second mode of operation were implemented, it wouldn't be necessary to use a maintenance interval at all. Since the second mode has very low overhead (no index-wide sweeps) and would only be invoked if it was known that work needed to be done, this infrastructure could be used to schedule tasks at arbitrary times instead of at the maintenance interval. The only necessary change would be to add a parameter to the task registration method containing a timestamp. The task would be invoked while processing the first block at or following the supplied timestamp.