Open joshburkart opened 3 weeks ago
A subtle topic indeed!
TL;DR: it is not possible to ensure determinism with schedule_periodic_event
, but if you are willing to live on the bleeding edge this is possible with the development branch (main
) using the schedule
method and periodic Action
s.
In general, the scheduler attempts to strike a balance between efficiency and determinism. The minimum guarantee when it comes to determinism is that the execution order must be preserved within messages addressed to the same model. To achieve this, when events are polled from the scheduler queue (which preserves scheduling order), all events targeting the same model are fused into a single sequential Future
and this fused Future
is spawned onto the executor. However, events targeting different models will end up in different Future
s so even though they are spawned in order, there is no telling in which relative order the multi-threaded executor will run them.
In the development branch, EventSource
s were introduced which are basically free-standing Output
s that can schedule Action
s that broadcast the same message (possibly periodically) to many models. But since several Action
s scheduled for the same timestamp may have one or mode target models in common, in order to preserve the execution order within messages to the same model, the scheduler needs to be conservative and fuse all Actions
in a single sequential future.
You can take advantage of that by scheduling two periodic Action
s using two EventSource
s, one targetting only model A
and one targetting only model B
, relying on the fact that the scheduler will fuse the Action
s into a sequential future. While this is not documented behavior, I don't see that changing in the future. Note that you cannot use a single EventSource
that broadcasts to A
and B
because for efficiency reasons, all broadcast operations resolve futures in parallel so ordering is no guaranteed (similarly to futures::stream::FuturesOrdered
).
The scope for version 0.3 keeps on expanding so I certainly won't consider this for the next version, but some simulators have a concept of super-dense time where the timestamp has an additional integer part that make it possible to decompose a time step into several sub-steps. AFAIK this concept does not exist in the ECSS SMP
standard which suggests it might be a bit niche, but if you feel the need for that at some point, feel free to open a specific issue.
This issue prompted further internal discussions and we came to the conclusion that the current ordering guaranties for scheduling are not fully consistent with the ordering guaranties for non-scheduled messages (those sent from Output
or Requestor
ports).
What we should have done in retrospect is to guaranty ordered sequential execution for same-time events based on the model that scheduled them rather than based on the models targetted by such events. Doing this would provide exactly the same kind of determinism as for regular messages: if model A
schedules message M1
to model B
and then schedules message M2
to model C
which upon receipt of M2
sends message M3
to model B
, it is guaranteed that B
will process M1
before M3
.
In this context it makes sense to treat the main scheduler as a model, which would give exactly the determinism you expected, whether an event or an action is used.
This is thus added to the v0.3
roadmap, but it shouldn't be too difficult to implement this behavior in the 0.2
branch and it is very unlikely to break existing user code. Let me know if you need it now, we could try to make a point release.
That makes sense -- thanks so much! I don't need it urgently but definitely a desirable feature in the next 3 months or so for me.
I had a follow-up question: after the changes you described are implemented, under what conditions will an Asynchronix simulation be fully deterministic? Will it be sufficient for all constituent models to run deterministically and possess no shared state?
I'm wondering about determinism in the following simplified situation: Say we have two models,
A
andB
, each with anexecute
method that we schedule to run periodically at the same periodicity and no phase offset. Thus for each cycle,A::execute
andB::execute
will run at the same simulated time. Say further that theexecute
method forA
immediately sends an event toB
, which modifiesB
's state, affecting whatB
'sexecute
method will do. As a result, without a way of specifying which periodic executes first each cycle, there appears to be a race condition possible where the outcome of the simulation isn't fully determined/predictable.Am I understanding this right? Are there ordering guarantees for periodics? Is there a way to ensure determinism, other than something hacky like adding tiny phase offsets? Would appreciate any context that can be provided. Thanks so much!