Open domkm opened 7 years ago
In theory that possible, but it would take a bit of work to achieve. Right now the components are sorted and started serially; we'd need a more sophisticated system to figure out which could be started in parallel.
@weavejester how sophisticated would this be? I assume you would need to model the dependency as a directed acyclic graph, then traverse it, starting from nodes with no incoming edges, and at each step loading all selected nodes in parallel? Or am I missing something?
You'd also need to handle threading, Integrant might need to provide a thread pool, or let the user provide one. It seems possible, but would need some thought to keep it easy for the user.
The dependencies are already modelled as a graph, which can be retrieved with the dependency-graph
function. However, there are a few difficulties around initiating concurrently:
From Erlang Programming Rules:
3.12 Make code as deterministic as possible
For example, suppose one process has to start five parallel processes and then check that they have started correctly, suppose further that the order in which these five are started does not matter.
We could then choose to either start all five in parallel and then check that they have all started correctly but it would be better to start them one at a time and check that each one has started correctly before starting the next one.
Well, probably the default should be start serially.
Only if is really needed, it could explicitly be asked to start something in parallel.
My solution for this problem https://gist.github.com/serioga/1c2f5a6ef771a267a6e726a066e2bf39
Parallel initialization addon for integrant https://github.com/serioga/webapp-clojure-2020/blob/master/src/lib/integrant/async.clj
For example, component
A
depends onB
andC
but bothB
andC
are independent and can take quite some time to initialize. Could Integrant startB
andC
concurrently instead of serially?