We use listeners to track the state of incoming objects and completed validations.
Listener for received objects: each time a peer receives an object, it notifies the network, which in turn calls the provided callback of all subscribed listeners with the received object id.
Listener for validated objects: each time a peer validates an object, it notifies the network whether or not this validation succeeded. The network in turn calls the provided callback of all subscribed listeners who are awaiting validation of the given object id with the received object id, deleting the subscriber from its list. The callbacks restart validation, so if the validation gets paused again, a new subscription will be opened.
When a peer receives an object and has to validate it, it captures its current context in a function closure validateRecursively which contains, essentially, a copy of the peer limited to the lifetime of the function.
Inside this validateRecursively function we rely on another function to do the meat of the work: validateNetworkObject.
In validateNetworkObject, we check missing object IDs, then subscribe to the network for 5 seconds, waiting to see if all objects have been sent during this time.
❗ Note: I don't know if it's possible for there to be a time frame between when we determine missing objects and subscribe to listen to incoming objects for objects to "slip through". An alternative would be maintaining some sort of log of incoming objects where we maintain the last 5 seconds of state, though it's potentially equally clumsy.
After that, we check the db as well as the received object ids provided by the network for our missing objects. If any are still missing, we send an unfindable_object error and halt validation.
Otherwise, we return a set of objectIds whose validation we are awaiting as well as a callback to re-run this validation.
The validateRecursively function checks the return value of validateNetworkObject and:
(1) if it returns a validity status, we're done and the object has been validated. validateRecursively determines an action for the peer to take, takes it and halts execution, releasing its closed values.
(2) If validation could not be finished, we do a quick re-check of the DB to see if data didn't happen to be validated while we were waiting and if so, we immediately re-run valiation with the callback provided by validateNetworkObject and pass the result to a recursive call to validateRecursively.
(3) If the re-check yields nothing, we suspend validation and let the network know by providing a callback with our closed over state which re-starts validation by running validateNetworkObject and then validateRecursively with the result. We then exit the function, handing off the closure to the callback maintained by the network.
Recursive validation is quite a doozy.
High level overview
We use listeners to track the state of incoming objects and completed validations.
validateRecursivelywhich contains, essentially, a copy of the peer limited to the lifetime of the function. Inside thisvalidateRecursivelyfunction we rely on another function to do the meat of the work:validateNetworkObject.In
validateNetworkObject, we check missing object IDs, then subscribe to the network for 5 seconds, waiting to see if all objects have been sent during this time.❗ Note: I don't know if it's possible for there to be a time frame between when we determine missing objects and subscribe to listen to incoming objects for objects to "slip through". An alternative would be maintaining some sort of log of incoming objects where we maintain the last 5 seconds of state, though it's potentially equally clumsy.
After that, we check the db as well as the received object ids provided by the network for our missing objects. If any are still missing, we send an
unfindable_objecterror and halt validation. Otherwise, we return a set of objectIds whose validation we are awaiting as well as a callback to re-run this validation.The
validateRecursivelyfunction checks the return value ofvalidateNetworkObjectand: (1) if it returns a validity status, we're done and the object has been validated.validateRecursivelydetermines an action for the peer to take, takes it and halts execution, releasing its closed values.(2) If validation could not be finished, we do a quick re-check of the DB to see if data didn't happen to be validated while we were waiting and if so, we immediately re-run valiation with the callback provided by
validateNetworkObjectand pass the result to a recursive call tovalidateRecursively. (3) If the re-check yields nothing, we suspend validation and let the network know by providing a callback with our closed over state which re-starts validation by runningvalidateNetworkObjectand thenvalidateRecursivelywith the result. We then exit the function, handing off the closure to the callback maintained by the network.