How to write Reactions to allow tiling (multiple tiles per Domain, defined by CellRange) and threads:
Any calculation of eg totals or averages that requires some kind of sum (a reduction) should define a global scalar variable with attribute :atomic=>true, and then work in two steps:
accumulate a "local" subtotal-like quantity for this tile into a local scalar variable
add to the global scalar by using PB.atomic_add!(global_scalar_var, subtotal)
The common case of calculating Domain totals eg for diagnostic output can just use add_method_do_totals_default! which creates _total Variables and adds a method to calculate them
If a method needs a temporary array buffer, then this should be defined per-thread eg as an Array of length Threads.nthreads() and the thread-specific buffer accessed as bufarray[Threads.threadid()]
If a method doesn't support tiling (eg it uses Domain averages/totals that are not calculated as above), then it should check length(cellrange.indices) == PB.get_length(m.domain) and error
How to define Domain tiling and configure solvers
Create a Vector of cellranges created with get_tiled_cellranges
For a single-threaded solver, this can be used just as a way of increasing memory efficiency
For a multi-threaded solver:
call create_modeldata with threadsafe=true when creating ModelData struct, subsequent calls to allocate_variables! will then check the :atomic attribute and create the appropriate Julia atomic variables.
call initialize_reactiondata! with method_barrier, a ReactionMethod created by reaction_method_barrier. A thread barrier will then be added inbetween each independent group of Reactions in the sorted list to synchronize threads.
Two pieces to this:
How to write Reactions to allow tiling (multiple tiles per Domain, defined by CellRange) and threads:
add_method_do_totals_default!
which creates _total Variables and adds a method to calculate themThreads.nthreads()
and the thread-specific buffer accessed as bufarray[Threads.threadid()]length(cellrange.indices) == PB.get_length(m.domain)
and errorHow to define Domain tiling and configure solvers
get_tiled_cellranges
create_modeldata
withthreadsafe=true
when creating ModelData struct, subsequent calls toallocate_variables!
will then check the:atomic
attribute and create the appropriate Julia atomic variables.initialize_reactiondata!
withmethod_barrier
, a ReactionMethod created byreaction_method_barrier
. A thread barrier will then be added inbetween each independent group of Reactions in the sorted list to synchronize threads.