search

keybase / kbpgp

OpenPGP (RFC4880) Implementation in IcedCoffeeScript
https://keybase.io/kbpgp
BSD 3-Clause "New" or "Revised" License
534 stars 74 forks source link

Support for streams #156

Open CaptnH00k opened 7 years ago

CaptnH00k commented 7 years ago

In your documentation you mention, that streams will be supported in the future ("For arbitrarily large files, streams will come soon in kbpgp's future."). Any ETA/progress/anything on this?

maxtaco commented 7 years ago

IT turned out to be insanely complicated, and soured us on PGP

kellym commented 7 years ago

I've been looking in the past for a way and just revisited this topic about an hour ago. There's definitely a need for stream support. Any (even brief) feedback on complexities or hurdles, or advice to abandon ship on writing it? I need to find a way to use streams to implement it since we have to do it on some huge files, but I'd love to not go down a rabbit hole if at all possible.

kellym commented 7 years ago

Nevermind. Found a fork of openpgpjs that supports streams and I see what you mean :). Well, I've got somewhere to start from at least.

maxtaco commented 7 years ago

I recommend saltpack instead.

CaptnH00k commented 7 years ago

I feared this answer, but it seems to be the same reason why openpgpjs didn't implement it yet. Still @kellym, which fork did you find that supports streams? Only know there was a pr for openpgpjs which kinda implemented it, but only for encryption if i remember it correctly. PR was this one: https://github.com/openpgpjs/openpgpjs/pull/321

Thanks for the saltpack recommendation, looks promising, but sadly no javascript.

maxtaco commented 7 years ago

We have a js implementation. Cc @oconnor663

maxtaco commented 7 years ago

The problem you are going to hit with steaming is that you have to create new sub streams in flight. For instance you decrypt, and you say oh shit, I have to start inflating now because there is a zlib stream inside the encrypted stream. It is an utter mess.

kellym commented 7 years ago

@CaptnH00k Yes, that openpgpjs PR was the one I had initially seen.

@maxtaco Ah, I haven't done enough digging in to come across that issue, that's a great point. We may end up having to create a custom solution then, since we only require encryption.

kellym commented 7 years ago

FYI @CaptnH00k, I opened a PR https://github.com/openpgpjs/openpgpjs/pull/567 that hopefully improves upon the existing streaming versions out there. It's still incomplete, but more complete than the alternatives.

Wish I could've done it for kbpgp instead, but I was deterred by the use of IcedCoffeeScript... I've been using CoffeeScript for what feels like ages but didn't feel like adopting ICS when everything else is moving to straight ES6.

HR commented 4 years ago

As of 2020, any plans to still add support?