search

ssbc / ssb-validate2-rsjs

Cryptographic validation of Scuttlebutt messages in the form of bindings (Rust) for Node.js.
3 stars 0 forks source link

Support toKVT or similar #26

Closed arj03 closed 3 years ago

arj03 commented 3 years ago

Often the messages you want to validate come over the network which means they don't have key or timestamp, only value. ssb-validate has a nice helper function for this. Since this library supports batches, the method should also support that I'm thinking. Not sure how much the overhead of rust is. One could consider maybe adding a validate single that can work on value directly instead of expecting message object?

What do you think @mycognosist?

mycognosist commented 3 years ago

@arj03

I see two options here (which are not mutually exclusive):

1) Support batch validation of message values (could be in addition to the existing KVT validation methods) 2) Implement the to_kvt helper and pass all messages values through it en route to the existing KVT validation methods

I'd be interested to hear from your preference is.

arj03 commented 3 years ago

A good question. The flow is we get some messages over the network and validate them, but then we need the key to insert them into the database. I'm not sure what the fastest option is, maybe we can try out option 2 first?

mycognosist commented 3 years ago

Sounds good. This will require some careful thought to avoid multiple data transformations across the Rust <-> JS boundary. The current way you're playing with OOO batch validation would pass the msgs back and forth twice: once for toKvt and once for validateOOOBatch. That's likely to reduce performance significantly.

I'm tinkering on this problem today, which should help me to develop a clearer overview of all the pieces. Will be good to have a call tomorrow to discuss the various trade-offs.

arj03 commented 3 years ago

Great thanks @mycognosist. Maybe we can use the cb based change that Andrè did combined with 1 to return the keys if things are valid?

mycognosist commented 3 years ago

@arj03 I think that's a great suggestion 👍🏻

mycognosist commented 3 years ago

@arj03

Are you happy if the batch validation functions return a serialized (stringified) array of JSON messages (in the form of key-values)?

arj03 commented 3 years ago

@mycognosist sure that could work. If its faster returning either a stringified array of keys or the keys directly as an array could also work, assuming the order is the same ;)

mycognosist commented 3 years ago

Oh cool, yeah it's much simpler (and likely faster) to return the keys without the associated values :) I can definitely return a stringified array of keys in order. Let me check about returning a JS array directly.

arj03 commented 3 years ago

@mycognosist can this be closed now? :)

mycognosist commented 3 years ago

Yes! :)