Finalizing the new entropy

pqwy commented 9 years ago

This is a tracking issue.

TL;DR: nocrypto has its own routines to init entropy on various platforms and (sometimes) has a hard dep on supporting code. To get fully seeded RNG on Unix, depend on nocrypto.lwt and call Nocrypto_entropy_lwt.initialize; to get it on Xen, depend on nocrypto.xen and call Nocrypto_entropy_xen.initialize. These routines can and should be invoked through mirage-the-tool.

what's there now

A V1.ENTROPY interface which purports to provide entropy and two implementations, mirage-entropy-unix and mirage-entropy-xen. On pure Unix, there's an equivalent but incompatible bit of code in tls.

what's the problem

A deep decision in the nocrypto-x509-tls chain is to treat RNG as an ambient effect. Type-wise, this means that the crypto protocols don't carry around RNG tokens, and operationally, this means nocrypto has a global RNG instance.

The problem here is that priming the RNG is a side-effect, so people can easily forget to request their V1.ENTROPY instance, without the typing structure forcing them to do so. Plus, as connecting the RNG to an entropy source is a global effect, it's not clear who should do it.

Another problem is that the entire concept of an entropy device is a bit wrong because the interaction between an RNG and its source is different depending on the nature of that source.

The final problem is that we lacked reliable entropy sources on xen.

what changed

nocrypto can arrange reseeding when running on lwt or xen.

lwt backend reads from the system provided entropy device, relies on openfile and the Lwt_main event loop and can be used on both pure Unix and Mirage-on-Unix.
xen backend uses inter-event timings and the CPU RNG if available. xentropyd code has been commented out but can be reinstated with some modifications.

Lwt code lives here, in nocrypto.lwt, because it is straightforward and more broadly applicable. Xen code lives in nocrypto.xen and defers to mirage-entropy-xen. (The current version of which, incidentally, works on Unix too but the expectation is that it will regain xentropyd support and really become Mirage-on-xen specific).

mirage-entropy-unix no longer conforms to a fixed V1 type. It is still a public library and can be used by anyone, but it precisely represents the entropy sources available within a unikernel running on Xen and nocrypto.xen has hard dependency on it.

As of right now, the mirage-entropy repo still contains code for mirage-entropy-unix and mirage-entropy-xen but I am about to delete the former and clean up the build system.

release?

[x] Merge the nocrypto branch.
[x] Merge the mirage-entropy branch.
[x] Merge the tls branch.
[x] Extend the mirage tool to detect direct or indirect usage of nocrypto, inject the appropriate dependency depending on the target platform and generate the init preamble. (here)
[x] Support ARM counters.

I'm currently working on all of the above.

Hope I remembered to write everything. Comments and feedback are more than welcome. Notably, the current design goes against the pervasive Mirage architecture where all code is parameterized by its dependencies, but after a painfully long deliberation, I think this strikes the right balance of simplicity and flexibility.

samoht commented 9 years ago

Hope I remembered to write everything. Comments and feedback are more than welcome. Notably, the current design goes against the pervasive Mirage architecture where all code is parameterized by its dependencies, but after a painfully long deliberation, I think this strikes the right balance of simplicity and flexibility.

I think that's indeed the right design choices.

hannesm commented 9 years ago

I read through the nocrypto and mirage-entropy-xen changes, and am in favour of merging them rather sooner than later. Would be nice to get it up and running on the cubieboard as well (cycle counter, there's even some fancy rng according to their documentation).

avsm commented 9 years ago

This looks very good. The main question remaining is at which point the RNG will give up due to not having enough quality crypto sources. This will presumably be hardcoded within the module and not parameterisable, so how is it currently chosen?

pqwy commented 9 years ago

@avsm Following what is my current blueprint for entropy harvesting in VMs, Whirlwind, the assumption is that a relatively accurate cycle-counter is present on the CPU and that the entropy gathered through that -- given the way it is collected -- is sufficient. There is basically no failure mode (1). The extra CPU features are probed at module-init-time and activated if present, and the expectation is that xentropyd will be probed for at connect time. Init will complete and the RNG will be seeded even if there are no optional features.

This is because I don't expect anyone to put too much thought into configuring their entropy sources. I, for one, really don't want to think about it when building my apps. So all the detection is done at runtime, with a baseline that always works.

If this baseline is deemed insufficient for a particular purpose, there is a way to query the entropy sources (API still a bit in flux) and bail out.

1: One thing I forgot yesterday: It needs a relatively widely available cycle-counting instruction for ARM here. @talex5 Do you know of something applicable? My knowledge of the ARM instruction set is shaky, and all my investigations led me to fairly vendor-specific things.

talex5 commented 9 years ago

I haven't used them, but the ARMv7 reference manual specifies that:

The HDCR.HPMN bit specifies the number of performance counters that the Guest OS can access. The minimum permitted value of HDCR.HPMN is 1, meaning there must be at least one counter that the Guest OS can access.

mirleft / ocaml-nocrypto