search

flumedb / flumelog-offset

MIT License
10 stars 8 forks source link

Async codecs #13

Closed christianbundy closed 5 years ago

christianbundy commented 6 years ago

I've been playing around with the unboxing code from secure-scuttlebutt and trying to implement some async unboxing code, but it looks like both the encoder and decoder need to be synchronous functions. Is it possible to use async functions for encode and decode, or would that be reasonable for me to [attempt to] implement?

Thanks!

dominictarr commented 6 years ago

Something like this would be really cool, because it leads towards private-groups... but I'm not sure wether that's the best way to implement them.

I am pretty sure though, that the next step is adding support for passing a decryption key link along with a link to an encrypted message... probably want to do that the same as secret blobs: https://github.com/ssbc/ssb-ref/pull/15 (and other issues starting out from there)

And also, figuring out exactly what the format for encrypted messages will be. That's the part with the highest technical barrier. hmm.

christianbundy commented 6 years ago

My goal was actually to start prototyping a blob content representation (like "content": "&sbBmsB7XWvmIzkBzreYcuzPpLtpeCMDIs6n/OJGSC1U=.sha256") rather than enc/dec, but from reading through the SSB code my intuition is that this may be like private messages, which need to be unboxed before they can be indexed. The problem is that multiblob's methods seem to be asynchronous will callbacks, but flumelog-offset expects a synchronous function without an option of using a callback.

I'm sure there are more high-performance ways of implementing off-chain messages, but I was hoping I'd be able to prototype it and get a better feel for the ergonomics (and deep-dive some SSB code).

Is there a better way of going about this?

christianbundy commented 6 years ago

FWIW, here's my current implementation, which works using fs.readFileSync():

https://github.com/ssbc/secure-scuttlebutt/commit/400d561e0f08461de2d790ed9d039b50eda417e2#diff-e91c0d50d79d0bc8fced372dd077f906R72

It's basically just a hacky and synchronous implementation of multiblob.get(), but I wanted to reach out for feedback if there's a simpler way to prototype this.