implement 'futures' for asynchronous computing?

[achubaty]

See here for a nice overview of how it could be used.

future package here: https://github.com/HenrikBengtsson/future

I can see this coming in handy for downloads, GIS operations, etc. which would normally block the main process -- these can be computed in a new process.

[achubaty]

even if we don't implement this in the package, this is likely still useful for module development.

[eliotmcintire]

We had talked about this before. I think it is necessary for us to implement in the main DES of SpaDES. Basically, when the DES comes across conditionally independent modules, it should automatically run them with the future function, so that two or more module events can run concurrently. The details of this will be somewhat complicated, as conditional independence will be sometimes difficult to establish. But in principle, it should be implemented.

Quote from here:

A more recent method is the three-phased approach to discrete event simulation (Pidd, 1998). In this approach, the first phase is to jump to the next chronological event. The second phase is to execute all events that unconditionally occur at that time (these are called B-events). The third phase is to execute all events that conditionally occur at that time (these are called C-events). The three phase approach is a refinement of the event-based approach in which simultaneous events are ordered so as to make the most efficient use of computer resources. The three-phase approach is used by a number of commercial simulation software packages, but from the user's point of view, the specifics of the underlying simulation method are generally hidden.

[achubaty]

Agreed (just wanted to document it here). It will take some work, but we already know the direct module dependencies so figuring out unconditionally co-occuring events should be doable. The indirect dependencies will be tricky to sort out though.

[eliotmcintire]

Easy types of module to deal with are summary-type modules and, more generally, modules with no-feedbacks. For ease of expressing the idea, we can call these branch tip modules. We can identify these modules with no feedbacks because no other module has inputs that match with these branch tip module outputs.

So, for example, a wildlife module in which all events require inputs from other modules (e.g., to take the habitat maps), but there are no feedbacks into the system (the other modules are not affected by any of the module outputs) because nothing that the wildlife does will affect any other modules.

These will be a good first type to deal with.

Having said that, I think I just worked through how to do it for indirect dependencies (see below).

I believe the work flow would be this:

• If one of these modules is scheduled to run, it's doEvent is spawned inside a future( ) call
• its valuation is forced at some time in the future (by calling value( )). I believe there is no actual requirement that it be valued at any particular time since its outputs are not required by anything else.
• Possible sensible times to call value would be at
  • end(sim) (the most computationally efficient because no other events will be conflicting at that point -- could be dangerous with infinite loop possibility, see next point)
  • at the next call to that same module-event (while not essential, this is aesthetically nice as it means we can't create an infinite loop)
  • in the case of indirect dependencies, we can assess dependencies at each doEvent and if the any preceding future call had an output dependency that matches one of the input dependencies, then it forces a call to value( ) before proceeding.

[achubaty]

This issue was moved to PredictiveEcology/SpaDES.core#18