rmacnak-google
commented
3 months ago While the GC could do better here, I think this is mostly from uneven work distribution across the isolates. I noticed the timeline often had fewer open HandleMessage events than the number of workers, especially toward the end. Here I've grouped the events by isolate instead of thread. (-j12, also --shards 4 --shard 1 because otherwise the trace is too big for Catapult to handle).
--- a/pkg/front_end/test/unit_test_suites.dart
+++ b/pkg/front_end/test/unit_test_suites.dart
@@ -6,6 +6,7 @@ import 'dart:async' show Timer;
import 'dart:convert' show jsonEncode;
import 'dart:io' show File, Platform, exitCode;
import 'dart:isolate' show Isolate, ReceivePort, SendPort;
+import 'dart:developer';
import 'package:args/args.dart' show ArgParser;
import 'package:testing/src/chain.dart' show CreateContext, Result, Step;
@@ -502,9 +503,11 @@ class SuiteConfiguration {
}
Future<void> runSuite(SuiteConfiguration configuration) async {
+ var t = new TimelineTask();
Suite suite = configuration.suite;
String name = suite.prefix;
String fullSuiteName = "$suiteNamePrefix/$name";
+ t.start(fullSuiteName);
Uri suiteUri = Platform.script.resolve(suite.path ?? "${name}_suite.dart");
if (!new File.fromUri(suiteUri).existsSync()) {
throw "File doesn't exist: $suiteUri";
@@ -528,6 +531,7 @@ Future<void> runSuite(SuiteConfiguration configuration) async {
shards: suite.shardCount,
shard: configuration.shard,
);
+ t.finish();
if (logger.gotFrameworkError) {
throw "Got framework error!";
}
jensjoha
mkustermann
On the CFE team we have quite a bit of "suites", to speed things up (and integrate with the approval system and whatnot) we have a "unit test suites" runner which launches the suites in isolates (using #cores-1 isolates).
My machine has 12 cores (with hyperthreading) and so it launches 11 isolates. According to
htopthough it only uses 4-500% CPU though. Prepending a run withtimegives this:(Runs on f774ec53e7b107664baf65a50395a717c3de26a2)
Using (user+sys)/real this becomes ~494% CPU.
If instead launching two processes, asking each to do half and only use 5 isolates each:
(run in two different terminals at the same time)
I get these numbers:
Using (user+sys)/real this becomes ~381% CPU.
and
Using (user+sys)/real this becomes ~374% CPU.
Adding up these numbers it becomes ~755% CPU (or ~747% if doing (user1+sys1+user2+sys2)/max(real1, real2)) --- in both cases quite a bit more than the <500% (more than 50% more actually). Looking at the (real) runtime it's almost twice as fast (~87% faster, taking ~53.5% of the time).
So: Are isolates expected to be slower, is it being held wrong or is something else up?
@mraleph suggested that "the answer is probably our GC does not scale well for this usecase" and suggested I file a bug, so here we are.