Sensitive data shouldn't get paged out to disk if possible

[thoughtpolice]

Talking with Dan Peebles, he mentioned someone else working on NaCl bindings (Michael Stone) brought up the fact that you should use something like mlock(2) on secret keys to ensure they're never paged to disk.

This would require changing the representation of the SecretKey type, although it needed consolidation anyway (it's merely aliased as ByteString in every module, which is dumb.) However, it will also break any sort of compiler portability at this point I think, as the underlying runtime will need to support something like pinned arrays. In GHC, a pinned array is a kind of ByteArray# that is never moved by the garbage collector, so foreign libraries can safely use it without fear of the pointer moving. So we can safely mlock(2) those pages afterwords.

We could possibly use a type like vector's Data.Vector.Storable type, which is pinned through a ForeignPtr. Generating random keys (via crypto_box_keypair or whatnot) would then be the process of allocating a pinned array (and maybe garbling it,) locking it, and then stashing the secret key to it - NaCl itself does no allocation, so it'll just cleanly write to the pointer we give it.

However, there are other concerns afoot in that if you were to call, say, readFile :: FilePath -> IO SecretKey on a secret key, it could potentially make a copy of that key somewhere in memory before returning a pinned, locked array. And that page could get paged out.

There may need to be GHC support for this ticket to really be sound/doable without mass FFI bindings. It will probably need an entire API dedicated to it anyway, since secret keys should especially be secure.

[tel]

Oh, this is far more thought out than my ticket. I really want this library pushed through and this is the sort of security critical concept that I'd love to see in an early release. It's a great point that the paging issue is somewhat significantly greater than just using Storables and mlocking—esp. w.r.t. getting the secret key into memory to begin with.

It might be a good first step to at least change the representation to Data.Vector.Storable still. I'll think about it.

[tel]

Generally, memory management doesn't exist at the level of a Haskell2010 API, so it'll might look like a contextual computation, perhaps

withKeyFile :: FilePath -> (Key -> IO a) -> IO ()
generateKey :: (Key -> IO a) -> IO ()

where I'm ()ing the wrapped IO values to indicate that keys cannot escape that context and are thus pinned and locked then unlocked, unpinned, and scrubbed (?) upon exiting—which isn't really guaranteed in IO, of course. An unsafe API could be exposed, too; a typesafe call down to mlock on keys so that people who have more complex locking behaviors aren't stuck.

[thoughtpolice]

One possibility is that because the Haskell2010 standard includes the FFI, we merely make FFI calls to functions like mlock(2) and other allocation functions directly. We also write our own FFI bindings to POSIX APIs to implement a very rudimentary file API in Haskell, perhaps like the one you have above.

In general I'm a fan of trying to actually write C as little as possible for two reasons: one, it's much easier to abstract over in Haskell with minimal bindings. Second, it's safer. In this case, an exception could potentially be made (that is, we just outright have a module that FFI's to a small body of C code, securely mlocking pages and writing/reading to/from disk securely. This is an open design question.

It may actually be reasonable to buy into GHC features for this specific purpose alone. We could reasonbly abstract out a 'simple, but not-paging-concious' API for more portability (e.g. windows, if that day ever comes.) The simple and portable API can just use readFile or whatever. The paging-aware API is more secure and defaulted to where applicable.

For example, a ByteArray# is pinned in the GC and we can allocate them using GHC primitives. These are of a guaranteed size and in a guaranteed location. So we could reasonably mlock a page-sized ByteArray# (the Addr# of which is constant,) and then 'carve' keys out of it to save memory.

So the process would be:

• Upon first calling a function like generateKey or withKey (or withKeyPair) or whatever for reading from a file, initialize by allocating a few pages of memory using mmap or somesuch behind the scenes.
• Lock these pages using mlock, ensuring they cannot be paged out.
• Garble these pages with data from /dev/urandom - there is already a nonblocking FFI call for this.
• Carve out enough space for this key in the set of pages, and return a pointer to it. This is the new Key type, which will then percolate into all the APIs
• Generate a random key and shove it into the carved out slab of space.
• Done.

For reading/writing from disk, the approach is similar, except we would want to first possibly use unbuffered IO to read directly from an fd into the ByteArray#. Whatever we do here, I think if we want to go this route we should offer an API for disk I/O for keys specifically.

This also has some complications that you would necessarily resize this set of pages when there's not enough space, etc. This module would then necessitate some local state to be able to safely generate and manipulate keys in a thread safe manner (because you'll need to manage the 'global cache of keys'.) Meaning we would need re-entrancy to be handled through a lock:

module Crypto.NaCl.Key (...) where

#ifdef __GLASGOW_HASKELL__ && defined(unix)
-- safe API

-- This is never exported.
keyStoreLock :: MVar Bool
keyStoreLock = unsafePerformIO $ newMVar False
{-# NOINLINE keyStoreLock #-}

#else
-- portable API for fallback
...
#endif

I think the first step is actually designing a portable, easy-to-use storage or key API that can be used for storage of keypairs and the like. It can be very simple I imagine in the general case. The necessity of keys being pinned can then be worked on in a more 'secure' implementation of the API.

This implementation I have outlined is something I have had in my head, not something I have put my money on yet. If that is too complicated, or there are better references concerning such problems, I'm open to alternative approaches.

[tel]

I agree that a stable key API is critical here.

The keystore management makes me think a lot about using the ST monad to control the pages in a less opaque way.

This might generally be a less important component of salt, though. If it's just made so that a key can be a pointer to some appropriate Storable then it becomes a separate concern to enable sensible manipulation of locked Storables and a few different strategies can be formulated.

[dnaq]

I believe that strict ByteStrings are actually pinned in memory, the reason being to lessen surprises when giving pointers to the ByteString contents to FFI-code that keeps the pointer (there was a thread on reddit about this issue recently)

To actually lock the key so that it isn't swapped is harder than just wrapping mlock(2) or VirtualProtect, since you need root access for mlock. You could always try to mlock, and log if you fail, but users of your library will probably not run their code as root. Application programs like GnuPG can start setuid root, lock their memory and then drop privileges, but that is hard for a library to do.

That being said, Vincent Hanquez has created a securemem library. As far as I know it doesn't do any locking, but it tries to scrub allocated memory when a pointer goes out of scope. I have created a branch in my fork of salt to use this library, and I could port it to your master branch if you are interested.