Correct way to reseed Fortuna.g

[objmagic]

Hi,

I am writing a module that provides nonce during OAuth. Here is my implementation so far.

The reason I need to remove alpha-numeric is stated in OAuth RFC and is irrelevant here.

I am not sure if I reseed the prng correct. Is there a more formal way to reseed prng?

[objmagic]

ping? @hannesm

[hannesm]

Since mirage-2.5.0 (released only to https://github.com/mirage/mirage-dev so far), the mirage tool emits the runes to seed and hookup the entropy sources - https://github.com/mirage/mirage/blob/master/lib/mirage.ml#L2443

Since nocrypto-0.4.0, you can use on unix: Nocrypto_entropy_lwt.initialize () (described https://github.com/mirleft/ocaml-nocrypto/blob/master/lwt/nocrypto_entropy_lwt.mli#L10, it will use /dev/urandom to periodically refeed during the lwt event loop), on xen Nocrypto_entropy_xen.initialize () (described https://github.com/mirleft/ocaml-nocrypto/blob/master/xen/nocrypto_entropy_xen.mli, it will use various available entropy sources (RDRAND/RDSEED/RTDSC; implemented in https://github.com/mirage/mirage-entropy) and refeed during the main event loop).

There is no need to manually create and reseed your Fortuna instance.

[hannesm]

skimming over rfc5849 I also cannot find the non-alpha-numeric condition you mention...

[objmagic]

Sorry, it is not specified in OAuth RFC, but required in Twitter dev document. See "Nonce" section

[hannesm]

thanks; they generate 32 bytes of random, and then base64encode them..

[objmagic]

yeah, that's what i did and I need to use re to remove equal signs, etc...

So, for Mirage, I do

module Random_mirage : Oauth.RANDOM = struct
  open Nocrypto

  let _ = Nocrypto_entropy_xen.initialize ()

  (** Create a Fortuna PRNG *)
  let prng = 
    let g = Fortuna.create () in
    Fortuna.reseed ~g (Cstruct.of_string "otter");
    g

 (* .......... *)

end

or the following one?

let prng = 
   Nocrypto_entropy_xen.initialize () >>= fun () ->
    return (Fortuna.create ())

[hannesm]

somewhere at top-level you should do a Nocrypto_entropy_xen.initialize () (only once!). then you can call let get_rand () = let r = Rng.generate 40 in ... no need for a custom Fortuna.

[objmagic]

thanks for your help! (* see u this summer in the 🐪lab *)

[hannesm]

and instead of the re stripping stuff, I'd then just use a hex encoding of the generated random... feels bad (at least for me) to strip an unknown amount of bytes out of your nonce...

[objmagic]

I agree. I feel unsafe b/c of this implementation. Maybe use this hex lib?

[pqwy]

@marklrh (and anyone else curious about using the RNG):

The overarching idea of randomness in nocrypto is that it's an ambient effect, which is a fancier name for a global variable. This makes it possible to simply call the functions in Rng and get your results with a minimum of fuss.

There is a global instance of RNG (currently a Fortuna.g but what exactly goes on inside is an implementation detail). As of fairly recently you can see it as Rng.generator. Every function that depends on randomness takes an optional parameter g to use as randomness source, and falls back onto this default instance if nothing is provided.

These g things can stretch a bit of randomness into an infinite stream, but it has to start somewhere. The initialization code, as a matter of convenience, arranges for periodic reseeding of whatever is the global generator at the moment it's called. There is no further logic around this, which means that if you swap the global generator you have to arrange its seeding, and if you called initialization prior to that, you also have to live with the fact that the program keeps on reseeding a dead generator.

So if you:

Are a library:

• Ignore initialization and leave that to the downstream application.
• Do not create new RNG instances yourself.
• Either call RNG functions completely ignoring the g parameter, of pass an optional one through. E.g.:

let f1 () = Rng.generate 17

let f2 ?g () = Rng.generate ?g 17

Are an application:

• If you are running lwt on Unix (incl. Mirage), call Nocrypto_entropy_lwt.initialize () (from nocrypto.lwt) before using the RNG.
• If you are running on Xen, call Nocrypto_entropy_xen.initialize () (from nocrypto.xen).
• If you are running on Unix but not using lwt, get in touch and we'll figure something out.

Note that it somehow happened that there are snippets of code floating around which do not wait for the initialization to finish and instead call it at the module level, like in your first example. Never do that, as it creates a race condition. There is a reason initialize returns unit Lwt.t as opposed to unit, surprise.

No:

let _ = Nocrypto_entropy_unix.initialize ()

let () =
  (* main *)
  ...

Yes:

let () =
  Nocrypto_entropy_lwt.initialize () >>= fun () ->
    (* main *)
    ...

The code will probably involve a call to Lwt_main.run in some fashion.

Are a MirageOS application:

As of 2.5.0, sit back and relax. mirage tool will do the equivalent of the above for you.

Are testing:

If you just wrote a small throw-away program, or loaded your stuff into a REPL and something keeps on throwing Fortuna.Unseeded_generator, do a simple one-time seeding:

let _ = Rng.reseed Cstruct.(of_string "completely unpredictable")

As soon as you apply reseed the generator considers itself well-seeded and you can continue. Most of my utop sessions start with the above, but make absolutely sure not to ship that in any serious code.

Know exactly what your are doing:

If you have strong opinions about RNG and/or seeding, and have intimately familiarized yourself with the RNG code, you can create your own Fortuna.g instances and use them either through the optional g parameter of relevant functions, or by swapping the Rng.generator. You can seed it either through calls to Fortuna.reseed, or better yet, attach an instance of Fortuna.Accumulator.t to your g and seed that, as it will maintain an entropy pool system for the generator.

When in doubt, don't.

Pitfalls

let _ = Nocrypto_entropy_xen.initialize ()

let prng = 
  let g = Fortuna.create () in
  Fortuna.reseed ~g (Cstruct.of_string "otter");
  g

The first call will schedule the initialization at some later time, depending on external events. If you were to use the global generator immediately following that call, you would still get an exception, because the rest of the program did not wait for the initialization to complete. (The exact result will actually wary depending on whether you are running on Unix or Xen.)

prng would be a new Fortuna.g which you could pass to various functions, but it would only ever be seeded with the string "otter", as the call to init arranged for the seeding of a separate generator -- the one in Rng.generator.

let prng = 
  Nocrypto_entropy_xen.initialize () >>= fun () ->
    return (Fortuna.create ())

This would create a lwt thread which, when completed, would get you an unseeded instance of g. As a side-effect, after completion of the thread, your Rng.generator would be properly seeded.

[pqwy]

As for emitting a sequence of symbols from a given alphabet, I would do this:

let sample alphabet =
  alphabet.(Rng.Int.gen_r 0 (Array.length alphabet))

let samplev alphabet n =
  let size = Array.length alphabet in
  Array.init n (fun _ -> alphabet.(Rng.Int.gen_r 0 size))

Modify as appropriate for the random-access structure containing your alphabet and the result sequence.

Note that this is not the most hi-performance way of doing things, as Fortuna will generate 32 bytes and perform rekeying for each place in the output. It is much better than generating a sequence of outputs in a batch and using mod to clip them to your alphabet size, however, as unlike with the modulo, you get a guaranteed flat distribution.

[objmagic]

@pqwy thanks!

I still have one question. For

let () =
  Nocrypto_entropy_lwt.initialize () >>= fun () ->
    (* main *)
    ...

For now, end-user has to do the above initialization

Is this design exposing the internal for user? I think it will be good if a user who wants to use my library doesn't need to know entropy generation or PRNG initialization stuff when he wants to have a quick start.

[objmagic]

@pqwy I changed my implementation to this as you suggested. Also, no more Cryptokit now.

Thanks a lot!

[pqwy]

The init in otter_test.ml is exactly what the top-level app needs to do.

Yes, it feels ugly. But I see no way to avoid it as there are two issues in play:

• Initial seeding of the RNG indeed is an effect. I don't see a way for a library to register a potentially blocking action to be fully completed on start of Lwt_main.run before any other processing has taken place. And even with that, nothing evaluated at module init time could use the RNG and making this explicit is less confusing. Keep in mind that init has to go out and open and read files and/or try to communicate across domains.
• Compositionality. If you try to pack it up inside your library, think about a user that happens to use several libraries that all depend on the same RNG.

I would really like to get rid of the requirement to call that preamble. As it is, forcing the application writer to arrange the start of their entropy seeding looks like the lesser of two evils.

Btw you can simplify Random_unix to:

let alpha =
  Array.init 62 @@ fun i ->
    char_of_int @@ i + (if i < 10 then 48 else if i < 36 then 55 else 61)

let get_nonce () =
  String.init 32 @@ fun _ -> alpha.(Rng.Int.gen_r 0 62)

Most of iter_follower in otter_test.ml is Lwt_stream.iter_s, too.

Anyway, looking forward to meeting you this summer. :smile: