Selectively firing rules, async & parallelism

[monkey-w1n5t0n]

Intro

On a scale of 1 to 10, where 1 is "just close your eyes and throw a pile of core.async at it!" and 10 is "you need a PhD in parallel systems and two decades of industry experience", how much do I have to think before I type to work with an odoyle session from multiple threads - and is it worth it? Are there any obvious gotchas, or any areas that may have some not-so-odvious gotchas and require more thought?

More specifically, I'm confused when it comes to thinking about facts being inserted vs rules being fired at different times, or whether it's possible (or even desirable) to selectively fire rules on specific threads. If it is, then do we have to ensure that the rules don't interact in any way, or can that still be managed?

I guess what I'm ultimately wondering about is whether rule firing can be selective and incremental, so that in an architecture like this:

          thread B
        /          \
thread A            thread D
        \          /
          thread C 

Thread A can fire only those rules that both B and C require, B and C fire independent rules in parallel, and thread D merges everything and carries on.

If it's possible, can the dependency graph be determined automatically, so that thread A knows which rules it has to fire given which rules threads B and C each want to fire themselves?

Example use case

Let's assume that I'm writing a TODO app with Vim-style modal editing and that I want something like the following async architecture, where each number is a loop running on a separate thread (real or core.async pool):

1. Receives keyboard and mouse events, timestamps them, and o/inserts them into some kind of queue in the session, without firing anything so that it doesn't block and miss(time) further incoming events.
2. Pulls from the event queue and fires only those rules that are triggered by those events that have been inserted, in the order they were inserted, with the state as it was at the time they were stamped. This presumably generates and inserts other facts, such as new TODO entries, TODO state changes, app mode changes etc.
3. Fires all other rules that may have been matched by the insertion of facts by no. 2, such as creating/modifying/deleting GUI components according to TODO entry creation/modification/deletion above, reinserting those into the session's GUI-related state.
4. o/querys the GUI-related state, topologically sorting them according to whether they overlap and passing each strand to be rendered in parallel by separate threads.

Example entries made by no. 1 (in chronological order & grouped by their relevance, see below):

{:key "n" :time 123}, {:key "esc" :time 124}, 

{:key "n" :time 125}, {:key "enter" :time 126}, {:key "tab" :time 127}

{:key "n" :time 128} {:key "A" :time 129}, {:key "B" :time 130}, {:key "enter" :time 131}

Example rules that are fired by no. 2 based on the above:

1. If the state is normal and the key n is pressed, make a new empty TODO, set it as current, and set the state to insert.
2. If the state is insert, the current TODO is empty, and the key esc is pressed, delete the TODO and set the state to normal.
3. If the state is insert and the key enter is pressed, save the current TODO and set the mode to normal.
4. If the state is normal and the key tab is pressed, set the current TODO's state to done.
5. If the state is insert and any other key is pressed, type the character in the TODO's text.

All of the above would result in:

1. The creation and instant deletion of an empty TODO (which happened so fast that it would never reach thread no. 3 and therefore no GUI component would be needlessly created)
2. The creation of an empty TODO, which is promptly marked as done so that I can pat myself on the back and take a well-deserved break to eat some cake.
3. The creation of a TODO with the text "AB".

More questions

1. Is this sort of thing even possible? Obviously an overkill for a TODO app, but I can imagine this sort of architecture being useful for apps like IDEs that would love to be as-asynchronous-as-possible and have this kind of declarative logic.
2. If it's not possible with a single session, then could it work with each thread having its own session and have a loop that looks like this:

   read events that previous thread left in the channel 
   -> insert them into own session 
   -> fire rules 
   -> do stuff with the results 
   -> put new facts on a channel for the next thread

   (I imagine this assumes that the rules for each thread's session don't have to interact, and that if they do that the state is duplicated and synced across the threads?)

3. What's the overhead in calling fire-rules? Is there a benefit in batching inserts before fire-rules is called, or would that cause problems? Would it help with avoiding unnecessary creation of GUI widgets, e.g. in the case where a TODO is created and then instantly deleted?

Outro

Thanks for writing and maintaining this awesome library, sorry for the barrage of questions and if half of them are obvious or don't make sense (or if they're a duplicate of #20 - that seemed more general than what I had in mind), I'd love to contribute to it at some point, writing async GUIs is hard, cheers & bye.

[oakes]

If you want to process state changes on separate threads but still cleanly merge their results together, I think you need a CRDT. They are designed for exactly that purpose: merging independent changes in a way that produces a deterministic result.

Rules can theoretically trigger any other rule, so I don't see a way to automatically separate them into silos that execute independently. If you managed to do this, it would sound quite similar to just using separate sessions with different rulesets.

To answer your question about the overhead of fire-rules, it is generally better to just call it once per "frame" rather than after each insert. In fact, I made an early mistake in pararules by making it fire rules automatically after every insert. You have to manually turn autoFire off to disable this behavior. In odoyle I don't even have an "auto fire" feature, because it's just wasteful and provides a really minor convenience.