Add batch validation for out-of-order (unordered) messages

[mycognosist]

Summary:

• Add validateOOOBatch(msgs)
• Add tests and a performance benchmark for validateOOOBatch(msgs)
• Wrap each batch verification / validation function in a helper which stringifies the input
  • This allows the library user to skip this step and simply pass in an array of message objects
• Update readme with behaviour details for validateOOOBatch(msgs)

Note: I'm using a fork of ssb-validate (rs) to provide the underlying OOO validation functionality.

Question(s):

validateOOOBatch() currently takes previous as a second, optional argument. This is a carry-over from validateBatch(). I think maybe we should remove support for previous in validateOOOBatch(), since it seems unnecessary? Another option is to run strict validation on previous when it is provided, such that msgs[1] must have sequence of msgs[0].sequence + 1 and the hash of msgs[0] must match msgs[1].previous. I suspect this might introduce unnecessary complexity for our use-case.

What do you think @arj03 ? Maybe just a simple 'validate this out-of-order batch' is good enough?

---

test/perf.js

# verifySignatures
ok 4 verified 10000 message signatures in 103 ms
ok 5 verified 10000 message signatures in 95 ms
ok 6 verified 10000 message signatures in 97 ms
ok 7 verified 10000 message signatures in 91 ms
ok 8 verified 10000 message signatures in 96 ms
ok 9 verified 10000 message signatures in 89 ms
ok 10 verified 10000 message signatures in 91 ms
ok 11 verified 10000 message signatures in 92 ms
ok 12 verified 10000 message signatures in 91 ms
ok 13 verified 10000 message signatures in 91 ms
average duration: 93.6 ms
# validateBatch
ok 14 validated 10000 messages in 127 ms
ok 15 validated 10000 messages in 143 ms
ok 16 validated 10000 messages in 127 ms
ok 17 validated 10000 messages in 128 ms
ok 18 validated 10000 messages in 125 ms
ok 19 validated 10000 messages in 128 ms
ok 20 validated 10000 messages in 129 ms
ok 21 validated 10000 messages in 127 ms
ok 22 validated 10000 messages in 129 ms
ok 23 validated 10000 messages in 135 ms
average duration: 129.8 ms
# validateOOOBatch
ok 24 validated 10000 messages in 133 ms
ok 25 validated 10000 messages in 137 ms
ok 26 validated 10000 messages in 136 ms
ok 27 validated 10000 messages in 135 ms
ok 28 validated 10000 messages in 136 ms
ok 29 validated 10000 messages in 147 ms
ok 30 validated 10000 messages in 136 ms
ok 31 validated 10000 messages in 135 ms
ok 32 validated 10000 messages in 135 ms
ok 33 validated 10000 messages in 133 ms
average duration: 136.3 ms
# appendKVT (legacy validation)
ok 34 validated 10000 messages in 563 ms
ok 35 validated 10000 messages in 556 ms
ok 36 validated 10000 messages in 550 ms
ok 37 validated 10000 messages in 551 ms
ok 38 validated 10000 messages in 553 ms
ok 39 validated 10000 messages in 552 ms
ok 40 validated 10000 messages in 550 ms
ok 41 validated 10000 messages in 551 ms
ok 42 validated 10000 messages in 551 ms
ok 43 validated 10000 messages in 551 ms
average duration: 552.8 ms

I'm not entirely certain why validateOOOBatch() is ever-so-slightly slower.

[arj03]

Yes, remove previous from validateOOOBatch that is just confusing. Great work on this PR! Very excited about this and that it has been tested to work for our specific needs.

[mycognosist]

Thanks @arj03 , really happy that it's starting to be useful. I've removed previous from validateOOOBatch() (much better).

[arj03]

This now looks good to me from an application developers perspective. I'm confortable enough with rust to review that. I'll say lets merge this.

[mycognosist]

Great, thanks arj. I imagine I'll get some feedback from Piet on the OOO Rust side of things when I open the issue on ssb-validate (rs).