service/network_profiling_test/service flakily produces RunTimeError on Windows configurations

[sstrickl]

There are new test failures on [ddc] Fix default type args signature on native classes...[vm] Tweak handling of rare types with non-null instance type arguments..

The tests

service/network_profiling_test/service RuntimeError (expected Pass)

are failing on configurations

vm-win-debug-x64c

It started being flaky on 23. February, with 62 passes and 57 RTEs, but has finally failed consistently enough to get promoted to RTE. Excerpt from the log:

--- Command "vm" (took 54.000536s):
set DART_CONFIGURATION=DebugX64C & set DART_SUPPRESS_WER=1 & set DART_CRASHPAD_HANDLER=C:\b\s\w\ir\out\DebugX64C\crashpad_handler.exe & out\DebugX64C\dart.exe --sound-null-safety -Dtest_runner.configuration=vm-win-debug-x64c --ignore-unrecognized-flags --packages=C:\b\s\w\ir\.dart_tool\package_config.json C:\b\s\w\ir\runtime\observatory\tests\service\network_profiling_test.dart

exit code:
255

stdout:
00:00 +0: network_profiling_test.dart (VM Service)

** Launching SERVICE_TEST_TESTEE=true out\DebugX64C\dart.exe --disable-dart-dev --write-service-info=file:///C:/b/s/w/it_s3zehou/dart_service2be99803/service_info.json --profiler --sound-null-safety -Dtest_runner.configuration=vm-win-debug-x64c --ignore-unrecognized-flags --packages=C:\b\s\w\ir\.dart_tool\package_config.json --enable-vm-service:0 C:\b\s\w\ir\runtime\observatory\tests\service\network_profiling_test.dart

>testee>out> The Dart VM service is listening on http://127.0.0.1:62444/_Ng-DO__GPU=/

>testee>out> 

** Signaled to run test queries on http://127.0.0.1:62444/_Ng-DO__GPU= (VM Service)

Loading VM...

Done loading VM

Running network_profiling_test.dart [1/5]

Running network_profiling_test.dart [2/5]

Running network_profiling_test.dart [3/5]

00:27 +0 -1: network_profiling_test.dart (VM Service) [E]

  Expected: <11>
    Actual: <8>

  package:matcher/src/expect/expect.dart 149:31                         fail
  package:matcher/src/expect/expect.dart 144:3                          _expect
  package:matcher/src/expect/expect.dart 56:3                           expect
  runtime\observatory\tests\service\network_profiling_test.dart 149:11  tests.<fn>.<fn>
  dart:core-patch/growable_array.dart 416:8                             List.forEach
  runtime\observatory\tests\service\network_profiling_test.dart 128:11  tests.<fn>
  ===== asynchronous gap ===========================
  runtime\observatory\tests\service\test_helper.dart 489:17             _ServiceTesterRunner.run.runTest.<fn>.<fn>
  ===== asynchronous gap ===========================
  runtime\observatory\tests\service\test_helper.dart 369:9              _ServiceTesterRunner.run.ignoreLateException
  ===== asynchronous gap ===========================
  package:test_api/src/backend/declarer.dart 215:9                      Declarer.test.<fn>.<fn>
  ===== asynchronous gap ===========================
  package:test_api/src/backend/declarer.dart 213:7                      Declarer.test.<fn>
  ===== asynchronous gap ===========================
  package:test_api/src/backend/invoker.dart 258:9                       Invoker._waitForOutstandingCallbacks.<fn>

** Killing script

00:28 +0 -1: Some tests failed.

Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
For example, 'dart test --chain-stack-traces'.

stderr:
Unhandled exception:
Dummy exception to set exit code.

--- Re-run this test:
python3 tools/test.py -n vm-win-debug-x64c service/network_profiling_test/service

The .../dds version is similarly flaky, so wouldn't be surprised if it also goes full RTE soon.

[dcharkes]

It's meta-flaking: https://dart-ci.web.app/#showLatestFailures=false&test=service/network_profiling_test/dds&configurations=vm-win-debug-x64c

Flipped from flaky to failing again today.

[derekxu16]

flipped again: https://dart-ci.appspot.com/log/vm-win-debug-x64c/vm-win-debug-x64c/208/service/network_profiling_test/dds

[sstrickl]

Looking at the Dart VM Flakiness dashboard, this is something that happens on all Windows bots, though apparently the debug bots are the only ones that RTE enough for it to flip from flaky to RTE instead of back to Pass.

Today it flipped on vm-win-debug-arm64:

...
00:20 +0 -1: network_profiling_test.dart (DDS) [E]

  Expected: <11>
    Actual: <8>

  package:matcher/src/expect/expect.dart 149:31                         fail
  package:matcher/src/expect/expect.dart 144:3                          _expect
  package:matcher/src/expect/expect.dart 56:3                           expect
  runtime\observatory\tests\service\network_profiling_test.dart 147:11  tests.<fn>.<fn>
  dart:core-patch/growable_array.dart 417:8                             List.forEach
  runtime\observatory\tests\service\network_profiling_test.dart 126:11  tests.<fn>
  ===== asynchronous gap ===========================
  runtime\observatory\tests\service\test_helper.dart 490:17             _ServiceTesterRunner.run.runTest.<fn>.<fn>
  ===== asynchronous gap ===========================
  runtime\observatory\tests\service\test_helper.dart 370:9              _ServiceTesterRunner.run.ignoreLateException
  ===== asynchronous gap ===========================
  package:test_api/src/backend/declarer.dart 229:9                      Declarer.test.<fn>.<fn>
  ===== asynchronous gap ===========================
  package:test_api/src/backend/declarer.dart 227:7                      Declarer.test.<fn>
  ===== asynchronous gap ===========================
  package:test_api/src/backend/invoker.dart 258:9                       Invoker._waitForOutstandingCallbacks.<fn>

** Killing script

00:20 +0 -1: Some tests failed.

Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
For example, 'dart test --chain-stack-traces'.

stderr:
Unhandled exception:
Dummy exception to set exit code.

--- Re-run this test:
python3 tools/test.py -n vm-win-debug-arm64 service/network_profiling_test/dds

The linked RTEs on the dashboard for the different configurations all seem to be consistent in sometimes getting 11 instead of 8 bytes written for https://github.com/dart-lang/sdk/blob/74eb7b38ea611007f09748102c4f0968bb75329c/runtime/observatory/tests/service/network_profiling_test.dart#L147

/cc @bkonyi