search

dart-lang / sdk

The Dart SDK, including the VM, JS and Wasm compilers, analysis, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
10.11k stars 1.56k forks source link

The test service/get_vm_timeline_rpc_test/service turned from flaky to Timeout on dartk-linux-release-ia32 #43504

Open aam opened 4 years ago

aam commented 4 years ago

The test service/get_vm_timeline_rpc_test/service turned from flaky to Timeout on dartk-linux-release-ia32

https://dart-ci.appspot.com/log/vm-kernel-linux-release-ia32/dartk-linux-release-ia32/8073/service/get_vm_timeline_rpc_test/service

mkustermann commented 3 years ago

This is still happening sometimes. After looking a little, I came to the following conclusions:

This test runs the subprocess with --complete-timeline. That will cause this test to consume a lot of memory, since all timeline events are collected.

On normal architectures, like X64, this test finishes relatively quickly, there are not so many timeline events.

%  measure out/ReleaseX64/dart runtime/observatory_2/tests/service_2/get_vm_timeline_rpc_test.dart                                                              [50/56]
[]
00:00 +0: get_vm_timeline_rpc_test.dart (VM Service)

** Launching SERVICE_TEST_TESTEE=true out/ReleaseX64/dart ...
...
00:14 +1: All tests passed!                                                                                        

** Process exited                                                                                                                                                                                                                     
        Memory: 750320 kB  -    0:16.20 real,   16.82 user,     4.41 sys

=> It takes 17 seconds and 750 MB of RAM.

On IA32 specifically this test takes much longer and also consumes more memory:

% measure out/ReleaseIA32/dart runtime/observatory_2/tests/service_2/get_vm_timeline_rpc_test.dart
[]
00:00 +0: get_vm_timeline_rpc_test.dart (VM Service)                                                               

** Launching SERVICE_TEST_TESTEE=true out/ReleaseIA32/dart ...

00:36 +1: All tests passed!

** Process exited
        Memory: 1463996 kB  -   0:40.82 real,   43.28 user,     7.86 sys

=> It takes 40 seconds and 1.5 GB of RAM. On DebugIA32 these numbers are worse.

The reason for this is that the test itself as well as the subprocess are launched from-source, that means it has to be compiled to kernel by the kernel-isolate.

On X64 the kernel-isolate is app-jit trained and very fast. Though on IA32 the kernel-isolate runs from a kernel file and needs to JIT a lot. That causes it to be slower and generate more timeline events, which the parent process then has to process.

One can reproduce that with X64 as well:

% measure out/ReleaseX64/dart --dfe=out/ReleaseIA32/gen/kernel_service.dill runtime/observatory_2/tests/service_2/get_vm_timeline_rpc_test.dart
...
** Killing script

00:38 +1: All tests passed!

** Process exited
        Memory: 2056548 kB  -   0:43.18 real,   45.72 user,     11.71 sys

Since we cannot do AppJIT training for IA32, one option would be to compile the script to kernel and launch the subprocess on this kernel file instead.

=> That would probably fix the problem.

aam commented 3 years ago

same just happened on https://ci.chromium.org/ui/p/dart/builders/ci.sandbox/app-kernel-linux-release-x64/11472/overview

mkustermann commented 3 years ago

I'll take a stab at trying to fix this as described in https://github.com/dart-lang/sdk/issues/43504#issuecomment-854744239

-> cl/210723