Document how to use Domain tiling and threads

[sjdaines]

Two pieces to this:

1. How to write Reactions to allow tiling (multiple tiles per Domain, defined by CellRange) and threads:

   • Any calculation of eg totals or averages that requires some kind of sum (a reduction) should define a global scalar variable with attribute :atomic=>true, and then work in two steps:
     1. accumulate a "local" subtotal-like quantity for this tile into a local scalar variable
     2. add to the global scalar by using PB.atomic_add!(global_scalar_var, subtotal)
   • The common case of calculating Domain totals eg for diagnostic output can just use add_method_do_totals_default! which creates _total Variables and adds a method to calculate them
   • If a method needs a temporary array buffer, then this should be defined per-thread eg as an Array of length Threads.nthreads() and the thread-specific buffer accessed as bufarray[Threads.threadid()]
   • If a method doesn't support tiling (eg it uses Domain averages/totals that are not calculated as above), then it should check length(cellrange.indices) == PB.get_length(m.domain) and error

2. How to define Domain tiling and configure solvers

   • Create a Vector of cellranges created with get_tiled_cellranges
   • For a single-threaded solver, this can be used just as a way of increasing memory efficiency
   • For a multi-threaded solver:
     • call create_modeldata with threadsafe=true when creating ModelData struct, subsequent calls to allocate_variables! will then check the :atomic attribute and create the appropriate Julia atomic variables.
     • call initialize_reactiondata! with method_barrier, a ReactionMethod created by reaction_method_barrier. A thread barrier will then be added inbetween each independent group of Reactions in the sorted list to synchronize threads.