search

tarantool / test-run

Tarantool functional testing framework
14 stars 15 forks source link

test-run sometimes doesn't kill server started by test #345

Open locker opened 2 years ago

locker commented 2 years ago

If a test starts a server, it should stop it on completion. However, if it doesn't, the server should still be stopped by test-run or luatest. Normally, this is what happens, but sometimes, the test server somehow survives.

How to reproduce:

  1. Revert https://github.com/tarantool/tarantool/pull/7359. The PR added server stop to vinyl-luatest/update_optimize_test.lua.
  2. Run the test in a loop: 
    yes vinyl-luatest/update_optimize_test.lua | head -n 30 | xargs ./test-run.py --builddir ../build/debug
  3. Check if there Tarantool instances left after the test: 
    ps ax | grep tarantool

(You'll probably need to try a few times before you catch it.)

A stray instance is running normally: it can be connected to or killed with SIGTERM.

Output ``` vlad@esperanza:~/src/tarantool/tarantool/test$ ps ax | grep tarantool 240930 pts/3 S+ 0:00 grep --color=auto tarantool vlad@esperanza:~/src/tarantool/tarantool/test$ yes vinyl-luatest/update_optimize_test.lua | head -n 30 | xargs ./test-run.py --builddir ../build/debug Started ./test-run.py --builddir ../build/debug vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua vinyl-luatest/update_optimize_test.lua Running in parallel with 16 workers Timeout options: ------------------- SERVER_START_TIMEOUT: 90 REPLICATION_SYNC_TIMEOUT: 100 TEST_TIMEOUT: 110 NO_OUTPUT_TIMEOUT: 120 Collecting tests in 'app' (Found 0 tests): application server tests. Collecting tests in 'app-luatest' (Found 0 tests): application server tests on luatest. Collecting tests in 'app-tap' (Found 0 tests): application server tests (TAP). Collecting tests in 'box' (Found 0 tests): Database tests. Collecting tests in 'box-luatest' (Found 0 tests): Database tests. Collecting tests in 'box-py' (Found 0 tests): legacy python tests. Collecting tests in 'box-tap' (Found 0 tests): Database tests with #! using TAP. Collecting tests in 'engine' (Found 0 tests): tarantool multiengine tests. Collecting tests in 'engine-luatest' (Found 0 tests): Database tests. Collecting tests in 'engine-tap' (Found 0 tests): tarantool multiengine tap tests. Collecting tests in 'engine_long' (Found 0 tests): tarantool engine stress tests. Collecting tests in 'long_run-py' (Found 0 tests): long running tests. Collecting tests in 'replication' (Found 0 tests): tarantool/box, replication. Collecting tests in 'replication-luatest' (Found 0 tests): replication luatests. Collecting tests in 'replication-py' (Found 0 tests): tarantool/box, replication. Collecting tests in 'small' (Found 0 tests): libsmall unit tests. Collecting tests in 'sql' (Found 0 tests): sql tests. Collecting tests in 'sql-luatest' (Found 0 tests): SQL tests on luatest. Collecting tests in 'sql-tap' (Found 0 tests): Database tests with #! using TAP. Collecting tests in 'swim' (Found 0 tests): SWIM tests. Collecting tests in 'unit' (Found 0 tests): unit tests. Collecting tests in 'vinyl' (Found 0 tests): vinyl integration tests. Collecting tests in 'vinyl-luatest' (Found 30 tests): vinyl space engine luatests. Collecting tests in 'wal_off' (Found 0 tests): tarantool/box, wal_mode = none. Collecting tests in 'xlog' (Found 0 tests): tarantool write ahead log tests. Collecting tests in 'xlog-py' (Found 0 tests): legacy python tests. Tarantool server information | Found executable at /home/vlad/src/tarantool/tarantool/build/debug/src/tarantool | Found tarantoolctl at /home/vlad/src/tarantool/tarantool/build/debug/extra/dist/tarantoolctl | Tarantool 2.11.0-entrypoint-201-g1fa9b6648725 | Target: Linux-x86_64-Debug | Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=ON | Compiler: /usr/bin/cc /usr/bin/c++ | C_FLAGS: -fexceptions -funwind-tables -fno-common -fopenmp -msse2 -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-gnu-alignof-expression -fno-gnu89-inline -Wno-cast-function-type -Werror | CXX_FLAGS: -fexceptions -funwind-tables -fno-common -fopenmp -msse2 -std=c++11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror ====================================================================================== WORKR TEST PARAMS RESULT --------------------------------------------------------------------------------- [007] vinyl-luatest/update_optimize_test.lua [ pass ] [003] vinyl-luatest/update_optimize_test.lua [ pass ] [004] vinyl-luatest/update_optimize_test.lua [ pass ] [010] vinyl-luatest/update_optimize_test.lua [ pass ] [009] vinyl-luatest/update_optimize_test.lua [ pass ] [001] vinyl-luatest/update_optimize_test.lua [ pass ] [013] vinyl-luatest/update_optimize_test.lua [ pass ] [008] vinyl-luatest/update_optimize_test.lua [ pass ] [014] vinyl-luatest/update_optimize_test.lua [ pass ] [006] vinyl-luatest/update_optimize_test.lua [ pass ] [002] vinyl-luatest/update_optimize_test.lua [ pass ] [011] vinyl-luatest/update_optimize_test.lua [ pass ] [016] vinyl-luatest/update_optimize_test.lua [ pass ] [005] vinyl-luatest/update_optimize_test.lua [ pass ] [012] vinyl-luatest/update_optimize_test.lua [ pass ] [015] vinyl-luatest/update_optimize_test.lua [ pass ] [007] vinyl-luatest/update_optimize_test.lua [ pass ] [001] vinyl-luatest/update_optimize_test.lua [ pass ] [004] vinyl-luatest/update_optimize_test.lua [ pass ] [010] vinyl-luatest/update_optimize_test.lua [ pass ] [006] vinyl-luatest/update_optimize_test.lua [ pass ] [014] vinyl-luatest/update_optimize_test.lua [ pass ] [003] vinyl-luatest/update_optimize_test.lua [ pass ] [005] vinyl-luatest/update_optimize_test.lua [ pass ] [002] vinyl-luatest/update_optimize_test.lua [ pass ] [009] vinyl-luatest/update_optimize_test.lua [ pass ] [013] vinyl-luatest/update_optimize_test.lua [ pass ] [016] vinyl-luatest/update_optimize_test.lua [ pass ] [011] vinyl-luatest/update_optimize_test.lua [ pass ] [008] vinyl-luatest/update_optimize_test.lua [ pass ] --------------------------------------------------------------------------------- Top 10 tests by occupied memory (RSS, Mb): * 401.0 vinyl-luatest/update_optimize_test.lua (Tests quicker than 0.1 seconds may be missed.) --------------------------------------------------------------------------------- Top 10 longest tests (seconds): * 1.81 vinyl-luatest/update_optimize_test.lua --------------------------------------------------------------------------------- Statistics: * pass: 30 vlad@esperanza:~/src/tarantool/tarantool/test$ ps ax | grep tarantool 241803 pts/3 Sl 0:00 tarantool default.lua 242301 pts/3 S+ 0:00 grep --color=auto tarantool vlad@esperanza:~/src/tarantool/tarantool/test$ kill 241803 vlad@esperanza:~/src/tarantool/tarantool/test$ ps ax | grep tarantool 242497 pts/3 S+ 0:00 grep --color=auto tarantool ```