search

browserify / crypto-browserify

partial implementation of node's `crypto` for the browser
MIT License
656 stars 199 forks source link

beyond crypto-browserify: modular crypto #128

Closed dominictarr closed 9 years ago

dominictarr commented 9 years ago

When I started the crypto-browserify project it was a simple idea: A pure javascript drop-in replacement for node's crypto module that would run in the browser. I never really expected that we could get as feature complete as we are now - I can only take a small part of the credit for this.

At the time, getting node's crypto module to work in the browser seemed like the right idea, but in the process, I have learnt a great deal more about cryptography and now I can see we can do much better.

Edit for clarity: crypto-browserify is not the end of the story, I think we need a new project (in a new org) that aims to do bulletproof crypto, with clearly defined properties and no surprising edgecases.

node's crypto is based on buggy standards - SHA, RSA, AES, TLS are all buggy. SHA1 is now considered to weak to be used in new protocols. SHA256 has length extension bug. RSA is unsafe without padding. AES has many insecure modes! If we are interested in building new cryptosystems (i am) then I do not think we should do so on primitives that you must understand the internal details of to use safely - What we need is a new collection of quality primitives. Hashes without length extension, signing and encrypting where the requirement for a clean nonce or padding is made explicit via the interface. Security properties must be simply stated and clearly documented.

edit for clarity: nacl has solved this problem for the primitives. It's a selection of quality primitives which have the clearest behavior, and are easy to understand with no surprises. This makes protocols built on them is easier to audit. We should use nacl, but build protocols on top of it applying the same philosophy.

Also, building your cryptosystem directly from the primitives is unwise. Even with quality primitives, there are many ways to assemble them, not all safe. Applications often need the same features, secure-channels, authentication, Encrypted storage, asymmetric encryption. These cryptomodules are assembled from the primitives, and likewise, their properties should be simply stated and clearly documented. The design process for a crypto module or protocol should be taken very seriously. Agile does work for crypto. To design a crypto protocol I think you should begin with a survey of prior art, and summarize and document the security properties. This will give you a good idea for the properties that are achiveable, you'll probably think of some new ones too! Document these, and design a protocol that realizes them. Describe the behavior your protocol will have with different kinds of attacks on it. Now you may begin to write code.

Yes, this is basically the waterfall method. Big Design Up Front. However, I am not advocating designing whole systems like this - just modules. What this is really, is a design process optimized for auditability.

I think this should probably be in the form of markdown documents with english descriptions and pseudocode.
A discussion needs to result in a easy to read document - not a back and forth issue discussion (though that is important while producing the document). Also, clear protocol design documents will be useful to developers across languages, not just javascript.

To give a concrete example of the kind of design process I am advocating have a look at what I've done here while designing a secure channel protocol: first, revew of prior art: https://github.com/ssbc/scuttlebot/wiki/secure-private-channels:-the-good,-the-bad,-and-the-ugly then summary of desirable properties: https://github.com/ssbc/scuttlebot/wiki/desirable-properties-for-a-secure-channel then a description of a protocol that meets all those properties https://github.com/ssbc/scuttlebot/wiki/a-secure-secure-channel

Who is with me?

@calvinmetcalf @dcousens @jprichardson @lsegal @kyledrake @wolfeidau @nathan7 @ahdinosaur @substack @hij1nx @ralphtheninja @jbenet @mafintosh

dominictarr commented 9 years ago

also @pfraze and @kumavis

kyledrake commented 9 years ago

I'm not really an expert on this, but a review of the philosophy of NaCl may be illustrative here: https://github.com/cryptosphere/rbnacl

kyledrake commented 9 years ago

I mean really, it would be nice to be able to use NaCl in the browser, but that's the least practical thing I have ever said in my career writing software.

lsegal commented 9 years ago

I agree with the general idea here. Specifically, I wholeheartedly agree that caution is much more important than moving fast when it comes to security and crypto. Getting it right is paramount.

That said, perhaps I might be confused, but I feel like this discussion might be overloaded. IMO this discussion should clearly separate "designing a protocol" from "implementing a protocol". These are two very different things. Implementing AES, or SHA-x, or even TLS, should not require a design document. These protocols have already been designed. Implementing per the existing spec should be the priority, which isn't quite as hefty a process. There are plenty of good reference implementations for many of these. Sure, some of these implementations have bugs, but no software is perfect. Using these references should provide a solid basis for this library.

The mention of "designing protocols" gets me a little concerned-- and scared. Is the goal of this module really to design new protocols? I respect all the minds contributing to this project (especially @dominictarr for stewarding this library, thank you!), but "I can do this better" is usually someone's famous last words when it comes to cryptography. More importantly, from a selfish-I'm-a-user perspective, I'm explicitly uninterested in any new crypto protocol, regardless of how much more amazing it may be. My project requirements require (lack of better term) me to use SHA-256, MD5, TLS, and others, regardless of how broken they might be. If crypto-browserify were to stop supporting those protocols, I wouldn't be able to use this library anymore. If new protocols really is the goal, they should be implemented in a new library. I'd gladly support the work done in this new library! (though "selfish-me" would have little use for it, or time to contribute)

If I'm misunderstanding feel free to correct me (I hope I am...), but I want to see this project continue to implement existing crypto protocols (AES, SHA-1, SHA-256, MD5, TLS) per spec, not going out and inventing new specs. Doing this should not require a heavy handed design doc, as those already exist as RFCs and other documentation. I am, of course, all for doing a more stringent job of ensuring that these existing specs are followed properly.

dominictarr commented 9 years ago

@kyledrake nacl in the browser: https://www.npmjs.com/package/ecma-nacl yes nacl is an inspiration here.

@lsegal sorry yes I am talking about starting a new project; I did not mean to suggest changing crypto-browserify's at all, it's exactly what it should be. I'll edit my issue to make that more clear.

For the record, I'm also not interested in inventing new primitives (hashes, ciphers, key exchanges, etc), but I think there is a need to curate the best of those.

pfrazee commented 9 years ago

I'm in favor of having fewer sharp edges on my crypto libraries. It is similar to nacl, how do you plan to differ? Different primitives, different features?

calvinmetcalf commented 9 years ago

this would probably fall under some sort of crypto-browserifyable banner but here is as good a place as any to discuss it, some other interesting prior art include open whisper systems and their version of otr.

dominictarr commented 9 years ago

@pfraze we'll most probably use nacl. we don't really need more primitives but we do need higher level things such as a secure-channel, and authentication protocols.

jbenet commented 9 years ago

@dominictarr I generally agree very strongly with you sentiment here. We should help.

Relevant:

@kyledrake

it would be nice to be able to use NaCl in the browser, but that's the least practical thing I have ever said in my career writing software.

hahah

((Aside, one might be able make a NaCl NaCl binary so you can salt hashes with salt running on salt. ))

Important note: if i'll contribute time to this, please note i've moved on from language-or-platform-dependent programming. I'd be game for defining library interfaces that are language independent -- meaning, to be implemented in multiple languages -- as idiomatic to those languages (e.g. callbacks in js, etc). (no "oh this works in the browser, so that's enough"). I mention this because to "build things that last" we must think beyond what's in vogue today.

jbenet commented 9 years ago

@lsegal important to address your comment.

I hear you-- very much understand your concerns. but i think @dominictarr is not asking for new protocols, rather simple, clear, usable implementations of existing, well understood protocols. Most cryptographic primitives are given to users with clear labels of how to use them, and the guarantees they provide. These already very clear labels are just not enough. As both djb and agl have pointed out multiple times, we must have extremely simple abstractions, down to simple functions of "the right thing to do given what we know today". We must not have the user choose a particular construction with a particular cipher in a particular mode -- this invariably leads to failure, as few users follow eprint and the relevant mailing lists, to know about the latest attacks. We must give users abstractions as easy as:

package crypto

func Encrypt(sk Key, plaintext []byte) (ciphertext []byte) { ...}
func Decrypt(sk Key, cipher []byte) (plaintext []byte) { ... }

func Sign(sk Key, data []byte) (sig []byte) { ... }
func Verify(pk Key, data []byte, sig []byte) (ok bool) { ... }

or continue to fool ourselves.

dominictarr commented 9 years ago

@jbenet I think we need to clarify some of the meanings we are using here.

A well designed protocol (like crypto_box) can be treated like a primitive, but it's not really a primitive (it uses key exchange - curve25517, a cipher - Salsa20, and a mac - poly1305) but it combines it's primitives into a simple api (though it could have better documentation).

we don't need new primitives, but we do need new protocols. @jbenet I also need a secure-channel - part 2 of your two layer tls protocol - secure-scuttlebutt has it's own approach to part 1. But I would love to collaborate on the secure-channel part.

dominictarr commented 9 years ago

oops, didn't mean to close that

kumavis commented 9 years ago

cc @wanderer

kumavis commented 9 years ago

so how do we start? order some hexagonal stickers?

wanderer commented 9 years ago

One thing that would be really nice is a good standard implementation of ECIES. It would be awesome to have some standered secure transportation protocols. I'm working on implementing this with @subtly. I would love to see how it compares to @dominictarr secure-channel and ipfs's secure channel.

kumavis commented 9 years ago

I think this is really important. crypto is too inaccessible.

I agree with the prior-art review and design-first approach. I agree with the approach of drawing the lines of modules based on features/use-cases. I agree with @jbenet's strategy of laying out a language/platform-agnostic spec for crypto modules.

There are a lot of topics to cover here, and we'll need to grow a community to generate easy-to-understand + well-reviewed specs.

so how do we start? order some hexagonal stickers?

By which I mean, what is the first step -- spin up a github org, create repos for different use cases (secure storage, tls), and start compiling research and ideas?

dominictarr commented 9 years ago

@wanderer looks interesting - I will have a look at this. Do you do IRC? I might need to ask you questions.

@kumavis yeah! before we can have stickers or github org we need a name. I've been calling this "modular crypto" to myself... that overloads the meaning of "modular" in the context of cryptography. You could call it a "library" in the sense of books on a shelf... but I don't really want to put everything in one repo (rather have one protocol one repo). More like an indie record label that is a curated selection of works that have a certain vibe. so someone needs to pick a reasonable name.

kumavis commented 9 years ago

so someone needs to pick a reasonable name.

oh man i've been in so many of these threads this year

wanderer commented 9 years ago

@dominictarr yeah I actually lurk on #scuttlebutt ;)

kumavis commented 9 years ago
kumavis commented 9 years ago

Guild and Consortium do sound a bit nose-in-the-air, we should ideally evoke an open community

dominictarr commented 9 years ago

@kumavis "Audit Driven Crypto" "auditdrivencrypto" ADC.

"audit driven" = do the security audit first, like test driven development.

kumavis commented 9 years ago

@dominictarr I like it. by naming the org after an approach to crypto development, it invites others to take that approach with us.

kumavis commented 9 years ago

@dominictarr if you're decently happy with that name I would encourage you to start an org and we can get to the good stuff. can still bike shed and rename org later.

dominictarr commented 9 years ago

@jbenet we need a word for the nonconcrete version of a protocol. I guess the best is "abstract protocol". An "abstract protocol" is based on "abstract primitives" so a hash is just a "hash", specifying the properties but not the implementation. then the implementation (with concrete primitives selected) is the concrete protocol.

dominictarr commented 9 years ago

done. https://github.com/auditdrivencrypto @kumavis need a hexagon logo.

dominictarr commented 9 years ago

should probably involve a lock with a green tick

dcousens commented 9 years ago

@dominictarr +1 to all the above. Thanks for clarifying the intentions, I wasn't sure what you were proposing at first.

jprichardson commented 9 years ago

If we are interested in building new cryptosystems (i am) then I do not think we should do so on primitives that you must understand the internal details of to use safely - What we need is a new collection of quality primitives. Hashes without length extension, signing and encrypting where the requirement for a clean nonce or padding is made explicit via the interface. Security properties must be simply stated and clearly documented.

@dominictarr This is paramount. I completely agree.

@lsegal has some important points...

The mention of "designing protocols" gets me a little concerned-- and scared. Is the goal of this module really to design new protocols? I respect all the minds contributing to this project (especially @dominictarr for stewarding this library, thank you!), but "I can do this better" is usually someone's famous last words when it comes to cryptography.

I share this sentiment. Maybe due to the whole "avoid rolling your own crypto" mantra so often touted, but I can definitely say that it makes me uncomfortable. For the sake of discussion, I'm glad that @dominictarr clarified the usage of cryptographic "primitives" vs "protocols". It sounds like this discussion is all about designing new protocols? When it comes to proving the security of cryptographic protocols (not primitives), what is the proper approach? Perhaps someone can point me to some resources (outside of a cursory Google search) that would help to shed some light?

Since this new organization is primarily about curated cryptographic primitives and new protocols, will this organization include any code or is it all about design? Overall, It definitely seems like writing up draft documents/specs is a reasonable start.

pfrazee commented 9 years ago

@jprichardson let me throw my take on this

Dominic is working on the security features of https://github.com/ssb/secure-scuttlebutt, which involves authentication, private channels, and blob encryption. He's doing his research and this project is how that's congealed

in secure-channel you can see in the desired properties and prior art what the goals are and why we dont use TLS/SSH/etc as a matter of course.

The goal's not to design protocols, it's to make informed decisions and properly document them, but new protocols may result. What would worry me is choosing TLS because it's a standard. What may happen is we choose TLS because it's the right choice for the crypto-system

subtly commented 9 years ago

I'm in.

It might be good to start with something simple and iteratively bring the secure-channel draft into compliance with the format/process. I would add one thing -- authors have veto. I'm not sure how that would work but it would be great to have some system for building this process around the author rather than the author conforming to the process. Then figure out how to integrate the two. I suppose this would mean an effort would have two specifications -- the author's and a specification which conforms to this process. That is, that the process has been followed, but not necessarily by the author.

Writing clean and easy to use software is difficult, let alone integrating cryptography. I think a project which exposes cryptography as a useable system deserves as much appreciation as the underlying cryptography. Applied cryptography is an application and I feel like this discussion is about applied cryptography. It looks like there's already a name but something along the lines of 'Applied Cryptography' would be nice (assuming no copyright conflict exists).

I'd just like to say that TLS/X.509 are legit. If TLS was tweaked just a bit, and it's implementations were a bit easier to manage, we might not be having this conversation. TLS is an excellent example of just how difficult it is to build a system which doesn't define primitives.

@dominictarr I do IRC. I would be happy to help @wanderer to build libraries which helps facilitate systems like scuttle-butt, ipfs, ethereum, etc. At a base level, all of these projects need the same sorts of codes and terminology -- key management, encoding/decoding, portability, authentication, etc.

Re: AEAD http://competitions.cr.yp.to/caesar.html

dominictarr commented 9 years ago

@subtly names are a classic bikeshed, and Applied Cryptography is already the title of a classic textbook

@jprichardson absolutely. If we are gonna challenge the taboo of creating new crypto (protocols) then we have to do it right. In a few years of learning about cryptography (from casual to pouring over classic papers) I havn't really encountered very much theory about how to design a protocol at all. But, there is a lot written about design process for regular software engineering.

I think we can transpose some of those concepts into crypto. For example, you would design your module or application using test driven development - well, to design a crypto protocol the test is the audit. So, we can do audit driven development. What does that look like? I think we write up an abstract description of the abstract protocol with pseudo code (we don't need to talk about byte offsets, or specific primitives, just "hash", "box" or "signature") - we then describe the operation of that protocol, in it's correct operation, but also what happens in the case of a replay attack, a man in the middle attack, and key compromise. Describe what an eavesdropper, replayer, middleman, and later key compromizer can learn from the protocol. It's essential to know what behavior occurs in the various attacks or failure modes, and what information can be extracted.

During the process we may learn about other failure modes, or other properties, in that case, iterate.

To start, study prior art. This lets come up with list of what properties are possible/desireable. You may have some ideas for properties that no current property supports (an example is that no secure-channel doesn't let a replay attacker confirm the server key - even in curvecp!)

Most crypto protocols are not documented to this level of details - A description of the correct operation is common enough but I think we need the How, What, and Why to really be spelled out. To optimize the design process for auditing, like in test driven development you optimist for testability.

this is not rolling "your own crypto" it's rolling "our own crypto".

dcousens commented 9 years ago

@dominictarr has this discussion been realized in any way? Perhaps a new repository to trial out ideas about what it is we want to do?

ahdinosaur commented 9 years ago

@dcousens you can see the progress that followed from this discussion in auditdrivencrypto, dominictarr/secret-handshake-paper, dominictarr/secret-handshake, ssbc/secret-stack, dominictarr/pull-box-stream, and probably heaps more that i'm missing.

dcousens commented 9 years ago

Awesome! Thanks @ahdinosaur

I might close this issue then. @dominictarr please re-open if you feel it is relevant.

dominictarr commented 9 years ago

yes close this, we have created https://github.com/auditdrivencrypto

On Thu, Aug 13, 2015 at 8:23 AM, Daniel Cousens notifications@github.com wrote:

Awesome! Thanks @ahdinosaur https://github.com/ahdinosaur

I might close this issue then. @dominictarr https://github.com/dominictarr please re-open if you feel it is relevant.

— Reply to this email directly or view it on GitHub https://github.com/crypto-browserify/crypto-browserify/issues/128#issuecomment-130551370 .