performance isolates

[gmb119943]

From a code performance point of view, is it better to use isolate pools to send unrelated tasks to isolates for execution? Or is it possible to create a new isolate for each task without loss of code performance?

[a-siva]

This is going to depend on a number of factors

• what would be considered loss of code performance in your case? on a linux workstation starting up an app like dart2js takes about 3 to 4 ms, this time is going to be based on the size of the app
• by unrelated tasks I presume there would be no communication or message passing between the isolates? If an isolate is to do some work and return significant results back to the spawning isolate then the run method (https://api.dart.dev/stable/2.19.3/dart-isolate/Isolate/run.html) which returns results with no copying would be more efficient
• having a pool, does allow control over the number of isolates spawned, we do have some known limitations on the number of isolates that can run (please see https://github.com/dart-lang/sdk/issues/51254 and https://github.com/dart-lang/sdk/issues/51261)

[gmb119943]

The scenario is roughly the following. There is a set of unrelated tasks. For each task, a new isolate is created and uses the exit function to pass the result without copying. Would it be better to keep a ready pool of isolates in this case, or is the cost of creating an isolate always minimal (if the number of isolates is less than the maximum limit, more than 16 isolates cannot be created on my PC)?

[lrhn]

As @a-siva says, "that depends".

Which operation dominates the computation? And is it speed or memory which is more important?

If you use Isolate.run, you spend time creating a new isolate and sending the initial message. Then you do the computation. Then you copy the result back for free. And the isolate goes away when it's done and takes no more memory.

If you use an isolate pool, you spend no time creating an isolate, send the initial message, do the computation, then spend time copying back the result. And the isolate stays alive, taking up member, whether you use it again or not.

The sending of the initial message and doing the computation are fixed costs.

For small return values, not creating a new isolate is definitely faster. For large return values, creating a new isolate, but getting free return shipping, is definitely faster.

To find the cut-off point, you will have to measure your program. The start-up time of an isolate will most likely depend, at least a little, on the size of the program it's being spawned from. Even with fast isolate spawning and sharing of immutable data, there will be some setup to make space for global mutable variables, which exist per-isolate.

The one further risk of an isolate pool is that you may get less parallelization. If you have 10 isolates in the pool, and you run 20 tasks, it will at most run 10 of those at a time. With 20 isolates, it can hypothetically run twice as fast. If the user has 20 CPU cores to run on, they're not doing anything else, and the stars are just right. (And if you can spawn 20 isolates. If there is a limit on how many isolates one can create, then a pool can help avoiding that, but going too close to the limit might break other libraries which try to create their own isolates.)

But you can also use a growing isolate pool which creates new isolates so every concurrent request has its own isolate, then it reuses those isolates only when the computation is done.

Then there is the memory cost of keeping isolates alive when they aren't needed any more. (And that's when one starts considering garbage-collecting isolates if usage drops for a while, or keep a hard maximum number of isolates, and all the other considerations you'd have for resource pools in general.)

There will be some "pool maintenance" cost, but that's likely to be negligible compared to the actual computations.

And there needs to be a pool strategy, which is at least:

• How many isolates initially?
• How many isolates max?
• Are isolates GC'ed if usage drops?
• Can more than one computation run on the same isolate at the same time? (Only works if the operations are async.)
• If so, is there a limit on how many? (If yes, you'd queue further operations locally until an isolate becomes free. That adds latency.)
• (If the pool is really overloaded, should you run the occasional asynchronous computation in the current isolate? You'd assume that some code in the local isolate is waiting for results for all the pending computations, so running a computation locally could actually make it progress faster.)

All these decisions factor into how efficient the pool will be. So try, and measure. There is no one answer which fits all programs.

(One example of a load-balancing pool is, from the no-longer-maintained package:isolate, LoadBalancer. Whether it fits your goals depend on what those goals are.)