Discrete event simulation and custom event loop support

[multun]

Hello :wave:

I'd like to contribute a discrete event simulation library which seamlessly integrates with kotlinx coroutines. This issue is about making it possible to implement it using the public API only.

I made a proof of concept, which revealed a few obstacles:

• the EventLoop API is private, which prevents hooking into runBlocking. In my POC, runSimulation is an equivalent of runBlocking, reconstructed using the public API.
• the Delay API is required to properly hook into the delay function. This API is public, but marked as internal, which requires @OptIn(InternalCoroutinesApi::class)

Context

• We're building a publicly founded OSS railway simulator.
• As huge railway systems have many moving bits and pieces (signals, switches, ...), spawning a thread for every async process is not an option.
• We also cannot use kotlinx.coroutines as is, as these simulations must run much faster than realtime
• As far as we know, our best option is to implement a discrete event simulation
• The only user friendly discrete event simulation library around is SimPy, and it's in Python. Even SimPy does not reuse python's async concurrently primitives, such as locks and queues, and re-implements job state management, exception propagation, and more.
• It seems like the ability of kotlinx.coroutines to customize scheduling and the implementation of delays negates the need for such a massive amount of work, which enables reusing all the amazing tools kotlin coroutines provide

How discrete event simulations work

They typically have an event loop, a virtual clock, a queue of callbacks for the current time, and a sorted queue of events (a timeline). Each event has a time at which it should occur, and a callback.

The algorithm implemented in my proof of concept goes as follows:

• run all callbacks from the current time queue
• pop the next event off the timeline. if there is none, end the algorithm
• set the virtual clock to the time of this event, and run its callback
• loop to the beginning

[dkhalanskyjb]

Hi! It's cool that people are using coroutines in such a manner.

I looked through your POC and want to mention that what you're doing looks very much like what's going on in the test module: https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-test In essence, it's a way to run coroutines using controllable virtual time, with additional safeguards against the occasional code that runs in a forked-off computation. Maybe you could use that or borrow some ideas from there?

Now, we do warn people not to use the test module anywhere but in tests, as the results of using it in production can be surprising, brittle and unpredictable. The most glaring issue would be that the virtual time can easily get out of sync with the real time, leading to all sorts of confusing scenarios when you have to interact with some other systems. This is one of the reasons why Delay is marked as internal. However, given that you already abuse the notion of delay not to mean real time, there's not much else, it seems, that can go wrong.

[qwwdfsad]

Thanks for the interest! We are not ready to accept it as kotlinx.coroutines module and would like to see it as a standalone library.

We are open to discussion for further opening our API and creating additional extension points, so feel free to create issues specific to a particular internal API