search

rnpgp / rnp

RNP: high performance C++ OpenPGP library used by Mozilla Thunderbird
https://www.rnpgp.org
Other
198 stars 55 forks source link

HSM support (discussion) #855

Open dewyatt opened 5 years ago

dewyatt commented 5 years ago

Description

This is a discussion issue for planning out HSM support. My initial thoughts are:

ronaldtse commented 5 years ago

The plan sounds sane. On the other hand we will need to decide what it should interact with:

dewyatt commented 5 years ago

@ronaldtse Generally I think nowadays all vendors provide a PKCS#11 API module at least, so they should all be very similar apart from quirks and features. Although it has been a while since I did any dev in this area.

dewyatt commented 5 years ago

Looking at this more I think it might make sense to go all-in on p11-kit and forego the Botan PKCS#11 wrapper. p11-kit provides some nice arbitration for access by multiple components in the same process, which may make sense in a library.

dewyatt commented 5 years ago

Revised plan:

  1. Migrate away from Botan's FFI and start using the C++ API directly, at least for crypto. This is necessary for the below because there is no Botan FFI access to the PKCS#11 wrapper and I don't think it would make much sense to add it. This will have some impact on the rest of the codebase, since it will introduce exceptions that we're not currently handling. So there will be resource leaks in the event of an error, until we fix functions individually to use smart pointers etc.
  2. Use Botan's PKCS#11 wrapper (it looks so convenient!), users can use p11-kit's proxy module to deal with things like multiple calls to C_Initialize/C_Finalize in the same process. Maybe botan will use p11-kit in the future.

I was able to get RSA decryption offloaded to an HSM in rnp today. It's an awful hack but worked as a proof-of-concept. I did fight a while tracking down this bug: https://github.com/randombit/botan/pull/1979 but overall Botan's PKCS#11 stuff seems nice to work with so far.

ronaldtse commented 5 years ago

@dewyatt thank you for the great news about the working PoC! The revised plan looks reasonable, and the road ahead looks much clearer now.

dewyatt commented 5 years ago

It looks like this is going to be a relatively large effort that will touch a lot of the codebase. Converting any single portion of rnp's code from using Botan's FFI to using Botan's C++ interface has a bit of a cascading effect requiring other sections to do the same (like if I convert the RSA code, I'll need to do the RNG code, and everything that uses that, etc). Plus I'll need to be cautious about exceptions in order to properly maintain our current error handling scheme.

I have made some progress but I'd like to put this on hold and re-prioritize as follows:

  1. Google Test conversion (I've only put a little bit of work into this so far)
  2. GitLab conversion (this is pretty far along, but will likely need more work for better parallelism)
  3. This

My main reasoning here is that I'm getting concerned about our CI run times approaching the limit of 50 minutes. We typically only reach about 40 minutes on a single job, so we still have some breathing room but that gap is closing with every test we add. Plus, the feedback loop is frustratingly slow for developers.

@ronaldtse Does this sound OK to you?

ronaldtse commented 5 years ago

@dewyatt sorry for the late reply! Yes, it sounds perfectly reasonable. With GitLab we can even shave the setup time using a custom docker container.