search

dart-lang / sdk

The Dart SDK, including the VM, dart2js, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
9.94k stars 1.53k forks source link

standalone/io/stdout_stderr_non_blocking_test timeout on precompiled arm android #28426

Open zanderso opened 7 years ago

rmacnak-google commented 7 years ago

This might be a crash that is being misreported as a timeout (note exit code -15).

FAILED: precompiler-dart_precompiled release_arm standalone/io/stdout_stderr_non_blocking_test
Expected: Pass 
Actual: Timeout
CommandOutput[precompiler]:
CommandOutput[adb_precompilation]:

stdout:
Executing adb -s 069a6592003ba933 shell rm -Rf /data/local/tmp/precompilation-testing/test ; echo AdbShellExitCode:  $?
Stdout:
AdbShellExitCode: 0
ExitCode: 0
Time: 0:00:00.092340

Executing adb -s 069a6592003ba933 shell mkdir -p /data/local/tmp/precompilation-testing/test ; echo AdbShellExitCode:  $?
Stdout:
AdbShellExitCode: 0
ExitCode: 0
Time: 0:00:00.044605

Executing adb -s 069a6592003ba933 push out/ReleaseAndroidARM/dart_precompiled_runtime /data/local/tmp/precompilation-testing/dart_precompiled_runtime
Stderr:
3381 KB/s (61553036 bytes in 17.776s)
ExitCode: 0
Time: 0:00:17.859768

Executing adb -s 069a6592003ba933 push out/ReleaseAndroidARM/process_test /data/local/tmp/precompilation-testing/process_test
Stderr:
3682 KB/s (2237308 bytes in 0.593s)
ExitCode: 0
Time: 0:00:00.677838

Executing adb -s 069a6592003ba933 shell chmod 777 /data/local/tmp/precompilation-testing/dart_precompiled_runtime /data/local/tmp/precompilation-testing/process_test ; echo AdbShellExitCode:  $?
Stdout:
AdbShellExitCode: 0
ExitCode: 0
Time: 0:00:00.067121

Executing adb -s 069a6592003ba933 push /b/build/slave/vm-precomp-android-release-3-3-be/build/sdk/out/ReleaseAndroidARM/generated_compilations/precompiler/tests_standalone_io_stdout_stderr_non_blocking_test/out.aotsnapshot /data/local/tmp/precompilation-testing/test/out.aotsnapshot
Stderr:
2504 KB/s (3634512 bytes in 1.417s)
ExitCode: 0
Time: 0:00:01.500721

Executing adb -s 069a6592003ba933 shell /data/local/tmp/precompilation-testing/dart_precompiled_runtime --ignore-unrecognized-flags --packages=/b/build/slave/vm-precomp-android-release-3-3-be/build/sdk/.packages /data/local/tmp/precompilation-testing/test/out.aotsnapshot ; echo AdbShellExitCode:  $?
Stdout:
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
Hello
ExitCode: -15
Time: 0:08:00.004640

Command[precompiler]: DART_CONFIGURATION=ReleaseAndroidARM out/ReleaseAndroidARM/clang_x86/dart_bootstrap --snapshot-kind=app-aot --snapshot=/b/build/slave/vm-precomp-android-release-3-3-be/build/sdk/out/ReleaseAndroidARM/generated_compilations/precompiler/tests_standalone_io_stdout_stderr_non_blocking_test/out.aotsnapshot --use-blobs --no-sim-use-hardfp --ignore-unrecognized-flags --packages=/b/build/slave/vm-precomp-android-release-3-3-be/build/sdk/.packages /b/build/slave/vm-precomp-android-release-3-3-be/build/sdk/tests/standalone/io/stdout_stderr_non_blocking_test.dart
Took 0:00:02.901646

Command[adb_precompilation]: Steps to push precompiled runner and precompiled code to an attached device. Uses (and requires) adb.
Took 0:08:20.318918

Short reproduction command (experimental):
    python tools/test.py -mrelease -cprecompiler -rdart_precompiled -aarm -sandroid --use-blobs -t480 standalone/io/stdout_stderr_non_blocking_test
nex3 commented 7 years ago

@rmacnak-google I'm working on getting more gardening issues assigned. If you're not the right person to assign this to, please reassign to that person.

mkustermann commented 6 years ago

The same test just timed out in appjit configuration:

FAILED: app_jit-vm product_x64 standalone_2/io/stdout_stderr_non_blocking_test
Expected: Pass
Actual: Timeout
--- Command "app_jit" (took 409ms):
DART_CONFIGURATION=ProductX64 out/ProductX64/dart --snapshot=/b/build/slave/app-linux-product-x64-be/build/sdk/out/ProductX64/generated_compilations/app_jit/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.jitsnapshot --snapshot-kind=app-jit --reify-generic-functions --limit-ints-to-64-bits --ignore-unrecognized-flags --packages=/b/build/slave/app-linux-product-x64-be/build/sdk/.packages /b/build/slave/app-linux-product-x64-be/build/sdk/tests/standalone_2/io/stdout_stderr_non_blocking_test.dart
exit code:
0
stdout:
Hello
Hello
Hello
Hello
Hello
Hello
stderr:
Hello
Hello
Hello
Hello
Hello
Hello
--- Command "vm" (took 01:00.000200s):
DART_CONFIGURATION=ProductX64 out/ProductX64/dart --reify-generic-functions --limit-ints-to-64-bits --ignore-unrecognized-flags --packages=/b/build/slave/app-linux-product-x64-be/build/sdk/.packages /b/build/slave/app-linux-product-x64-be/build/sdk/out/ProductX64/generated_compilations/app_jit/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.jitsnapshot
exit code:
-15
diagnostics:
Process list including children: [5970]
Trying to capture stack trace for pid 5970
PID 5970 - process
TID 5970:
#0  0x00007f4b41d78404 pthread_cond_wait@@GLIBC_2.3.2
#1  0x000000000068ddc1 dart::Monitor::WaitMicros(long)
#2  0x000000000072883a dart::ServiceIsolate::Shutdown()
#3  0x00000000005eb2c6 dart::Dart::Cleanup()
#4  0x000000000086760d Dart_Cleanup
#5  0x000000000046e48e dart::bin::main(int, char**)
#6  0x000000000046eb46 main
#7  0x00007f4b40f9cf45 __libc_start_main
#8  0x000000000046d029 _start
TID 5971:
#0  0x00007f4b41079693 epoll_wait
#1  0x0000000000475105 dart::bin::EventHandlerImplementation::Poll(unsigned long)
#2  0x0000000000490ceb dart::bin::ThreadStart(void*)
#3  0x00007f4b41d74184 start_thread
#4  0x00007f4b41078ffd __clone
TID 5972:
#0  0x00007f4b41d78404 pthread_cond_wait@@GLIBC_2.3.2
#1  0x000000000068ddc1 dart::Monitor::WaitMicros(long)
#2  0x000000000084ce6a dart::BackgroundCompiler::Run()
#3  0x0000000000742b7f dart::ThreadPool::Worker::Loop()
#4  0x0000000000742a1e dart::ThreadPool::Worker::Main(unsigned long)
#5  0x000000000068d623 dart::ThreadStart(void*)
#6  0x00007f4b41d74184 start_thread
#7  0x00007f4b41078ffd __clone
stdout:
Hello
Hello
Hello
Hello
Hello
Hello
stderr:
Hello
Hello
Hello
Hello
Hello
Hello
--- Re-run this test:
python tools/test.py -m product -c app_jit standalone_2/io/stdout_stderr_non_blocking_test
===
dcharkes commented 5 years ago

This issue resurfaced today on arm on a build bot.

FAILED: dartkp-dart_precompiled release_simarm64 standalone_2/io/stdout_stderr_non_blocking_test
Expected: Pass
Actual: Timeout

--- Command "vm_compile_to_kernel" (took 01.000958s):
DART_CONFIGURATION=ReleaseSIMARM64 /b/s/w/ir/pkg/vm/tool/gen_kernel --aot --platform=out/ReleaseSIMARM64/vm_platform_strong.dill -o /b/s/w/ir/out/ReleaseSIMARM64/generated_compilations/dartkp/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.dill /b/s/w/ir/tests/standalone_2/io/stdout_stderr_non_blocking_test.dart --packages=/b/s/w/ir/.packages -Ddart.developer.causal_async_stacks=true

exit code:
0

--- Command "precompiler" (took 02.000722s):
DART_CONFIGURATION=ReleaseSIMARM64 out/ReleaseSIMARM64/gen_snapshot --snapshot-kind=app-aot-blobs --blobs_container_filename=/b/s/w/ir/out/ReleaseSIMARM64/generated_compilations/dartkp/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.aotsnapshot --sync-async --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages /b/s/w/ir/out/ReleaseSIMARM64/generated_compilations/dartkp/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.dill

exit code:
0

--- Command "remove_kernel_file" (took 13ms):
DART_CONFIGURATION=ReleaseSIMARM64 rm /b/s/w/ir/out/ReleaseSIMARM64/generated_compilations/dartkp/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.dill

exit code:
0

--- Command "vm" (took 08:00.000157s):
DART_CONFIGURATION=ReleaseSIMARM64 out/ReleaseSIMARM64/dart_precompiled_runtime --sync-async --ignore-unrecognized-flags --packages=/b/s/w/ir/.packages /b/s/w/ir/out/ReleaseSIMARM64/generated_compilations/dartkp/tests_standalone_2_io_stdout_stderr_non_blocking_test/out.aotsnapshot

exit code:
-15

diagnostics:
Process list including children: [9411]
Trying to capture stack trace for pid 9411
PID 9411 - process
TID 9411:
#0  0x00007ff0a6145404 pthread_cond_wait@@GLIBC_2.3.2
#1  0x000055a24ce683c8 dart::Monitor::WaitMicros(long)
#2  0x000055a24cecb4ca dart::ServiceIsolate::Shutdown()
#3  0x000055a24cd8fe0f dart::Dart::Cleanup()
#4  0x000055a24cb93567 dart::bin::main(int, char**)
#5  0x000055a24cb93bc9 main
#6  0x00007ff0a5369f45 __libc_start_main
#7  0x000055a24cb92029 _start
TID 9413:
#0  0x00007ff0a54466d3 epoll_wait
#1  0x000055a24cb996d5 dart::bin::EventHandlerImplementation::Poll(unsigned long)
#2  0x000055a24cbb8e9e dart::bin::ThreadStart(void*)
#3  0x00007ff0a6141184 start_thread
#4  0x00007ff0a544603d __clone

stdout:
Hello
Hello
Hello
Hello
Hello
Hello

stderr:
Hello
Hello
Hello
Hello
Hello
Hello

--- Re-run this test:
python tools/test.py -m release -a simarm64 -n dartkp-linux-release-simarm64 standalone_2/io/stdout_stderr_non_blocking_test
sjindel-google commented 5 years ago

This issue resurfaced again in https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8930421880132214384/+/steps/vm_tests_shard_1/0/stdout, (config dartk-vm release_simdbc64).

aam commented 5 years ago

and today https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8929297456745571920/+/steps/vm_tests_shard_1/0/stdout on vm-kernel-precomp-linux-release-simarm

mraleph commented 5 years ago

Based on stack traces I think this is the same bug that @mkustermann has fixed in 83ba3973683981286455869dd473439aa7a8d3ff - we are not propagating UnwindError correctly.

aam commented 5 years ago

Still persists(https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket.appspot.com/8928428248600968144/+/steps/vm_tests_shard_1/0/stdout), but perhaps duplicate of https://github.com/dart-lang/sdk/issues/35192

a-siva commented 5 years ago

I was able to reproduce this locally (gdb) p shuttingdown $1 = true (gdb) p port_ $2 = 405827945 and so the shutdown loop is not exiting.

It looks like the service isolate exited without running through the shutdown code.

sjindel-google commented 5 years ago

This is still failing https://ci.chromium.org/p/dart/builders/luci.dart.ci.sandbox/vm-kernel-precomp-bare-linux-release-x64/17