tarantool / luatest

Tarantool test framework written in Lua
https://www.tarantool.io/en/doc/latest/reference/reference_rock/luatest/luatest_overview/
Other
40 stars 10 forks source link

Unable to start a TAP test #208

Open ligurio opened 2 years ago

ligurio commented 2 years ago

Unable to start a TAP test, test-run failed with ('Unable to runluatest --version', {'returncode': 1}). However luatest is available in $PATH.

sergeyb@pony:~/sources/MRG/tarantool/build/test$ git submodule update --init --recursive 
Submodule path '../../src/lib/msgpuck': checked out '7f95b6fd7a5b928cfcbdab2d78bea88d1685821e'
Submodule path '../../src/lib/small': checked out '055458f1d45948f7d768e2499496926dcf78b26f'
Submodule path '../../test-run': checked out 'd16cbdc2702e7d939d5d34e41730fd83d2c65879'
Submodule path '../../test-run/lib/checks': checked out 'c97888c8d4a5ffb0255267d2d82fd60bae17916a'
Submodule path '../../third_party/luajit': checked out '8304ca95c6c5bbdaf3b42c5c18aa9d6e31ea56e1'
sergeyb@pony:~/sources/MRG/tarantool/build/test$ git describe 
2.10.0-beta1-347-g88332b354
sergeyb@pony:~/sources/MRG/tarantool/build/test$ luatest --version
luatest v0.5.6
sergeyb@pony:~/sources/MRG/tarantool/build/test$ ../../test/test-run.py --builddir=/home/sergeyb/sources/MRG/tarantool/build --vardir=/home/sergeyb/sources/MRG/tarantool/build/test/var app-tap/gh-2717-no-quit-sigint.test.lua
Started ../../test/test-run.py --builddir=/home/sergeyb/sources/MRG/tarantool/build --vardir=/home/sergeyb/sources/MRG/tarantool/build/test/var app-tap/gh-2717-no-quit-sigint.test.lua
Running in parallel with 16 workers

Timeout options:
-------------------
REPLICATION_SYNC_TIMEOUT: 100
TEST_TIMEOUT:             110
NO_OUTPUT_TIMEOUT:        120

('Unable to run `luatest --version`', {'returncode': 1})
sergeyb@pony:~/sources/MRG/tarantool/build/test$ 
ylobankov commented 2 years ago

Could you please remember, tarantool was built with ASAN enabled or not? For now, I can reproduce this bug only if tarantool was built with ASAN enabled.

ligurio commented 2 years ago

Could you please remember, tarantool was built with ASAN enabled or not?

Unfortunately no :(

sergos commented 2 months ago

Doesn't reproduce:

dev1.tarantool.org:test $ python3 ../../test/test-run.py --builddir=/home/s.ostanevich/workspaces/t.sergos2/build --vardir=./var app-tap/gh-2717-no-quit-sigint.test.lua                                                                   
Started ../../test/test-run.py --builddir=/home/s.ostanevich/workspaces/t.sergos2/build --vardir=./var app-tap/gh-2717-no-quit-sigint.test.lua                                                                                             
Running in parallel with 160 workers      

Timeout options:              
-------------------                     
REPLICATION_SYNC_TIMEOUT: 100            
TEST_TIMEOUT:             110                      
NO_OUTPUT_TIMEOUT:        120                     

Collecting tests in 'app'            (Found 0   tests): application server tests.                                    
Collecting tests in 'app-luatest'    (Found 0   tests): application server tests on luatest.                         
Collecting tests in 'app-tap'        (Found 0   tests): application server tests (TAP).                              
Collecting tests in 'box'            (Found 0   tests): Database tests.                                              
Collecting tests in 'box-py'         (Found 0   tests): legacy python tests.                                         
Collecting tests in 'box-tap'        (Found 0   tests): Database tests with #! using TAP.           
Collecting tests in 'engine'         (Found 0   tests): tarantool multiengine tests.                                 
Collecting tests in 'engine_long'    (Found 0   tests): tarantool engine stress tests.                    
Collecting tests in 'long_run-py'    (Found 0   tests): long running tests.
Collecting tests in 'replication'    (Found 0   tests): tarantool/box, replication.                                                                                                                                                        
Collecting tests in 'replication-luatest' (Found 0   tests): replication luatests.                
Collecting tests in 'replication-py' (Found 0   tests): tarantool/box, replication.                                                                                                                                                        
Collecting tests in 'sql'            (Found 0   tests): sql tests.                                                                                                                                                                         
Collecting tests in 'sql-tap'        (Found 0   tests): Database tests with #! using TAP.                            
Collecting tests in 'swim'           (Found 0   tests): SWIM tests.                                                  
Collecting tests in 'unit'           (Found 0   tests): unit tests.                                                                                                                                                                        
Collecting tests in 'vinyl'          (Found 0   tests): vinyl integration tests.                                     
Collecting tests in 'vinyl-luatest'  (Found 0   tests): vinyl space engine luatests.                                                                                                                                                       
Collecting tests in 'wal_off'        (Found 0   tests): tarantool/box, wal_mode = none.                              
Collecting tests in 'xlog'           (Found 0   tests): tarantool write ahead log tests.                             
Collecting tests in 'xlog-py'        (Found 0   tests): legacy python tests.                                         

Tarantool server information                              
 | Found executable at /home/s.ostanevich/workspaces/t.sergos2/build/src/tarantool                  
 | Found tarantoolctl at /home/s.ostanevich/workspaces/t.sergos2/build/extra/dist/tarantoolctl                       

 | Tarantool 2.10.0-beta1-347-g88332b3540                                                                            
 | Target: Linux-x86_64-Debug                                                                                                                                                                                                              
 | Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=ON                 
 | Compiler: /opt/rh/devtoolset-8/root/usr/bin/cc /opt/rh/devtoolset-8/root/usr/bin/c++                                                                                                                                                    
 | C_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-gnu-alignof-expression -fno-
gnu89-inline -Wno-cast-function-type -Werror         
 | CXX_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c++11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-invalid-offsetof -Wno-gn
u-alignof-expression -Wno-cast-function-type -Werror                                                                 

======================================================================================                               
WORKR TEST                                            PARAMS          RESULT                              
---------------------------------------------------------------------------------
---------------------------------------------------------------------------------                                                                                                                                                          
Top 10 tests by occupied memory (RSS, Mb):                                                                           

(Tests quicker than 0.1 seconds may be missed.)
---------------------------------------------------------------------------------                                    
Top 10 longest tests (seconds):            
---------------------------------------------------------------------------------                                    
ligurio commented 2 months ago

The problem is always reproducing on Tarantool build with enabled ASAN and missed ASAN suppressions:

[0] ~/sources/MRG/tarantool $ ./test/test-run.py --builddir=$(pwd)/build test/app-luatest/http_client_test.lua
Started ./test/test-run.py --builddir=/home/sergeyb/sources/MRG/tarantool/build test/app-luatest/http_client_test.lua
Running in parallel with 16 workers

Timeout options:
-------------------
SERVER_START_TIMEOUT:     90
REPLICATION_SYNC_TIMEOUT: 100
TEST_TIMEOUT:             110
NO_OUTPUT_TIMEOUT:        120

('Unable to run `luatest --version`', {'returncode': 1})
[0] ~/sources/MRG/tarantool $ ./build/src/tarantool --version
Tarantool 3.3.0-entrypoint-8-g4e7ce35c8b
Target: Linux-x86_64-Debug
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=TRUE
Compiler: Clang-13.0.1
C_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -fsanitize=address -fsanitize-blacklist=/home/sergeyb/sources/MRG/tarantool/asan/asan.supp -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror -g -ggdb -O0 
CXX_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -fsanitize=address -fsanitize-blacklist=/home/sergeyb/sources/MRG/tarantool/asan/asan.supp -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c++11 -Wall -Wextra -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror -g -ggdb -O0 
[0] ~/sources/MRG/tarantool $ 
Totktonada commented 2 months ago

This problem is about test-run. It should report stdout/stderr from luatest --version, but it doesn't. See here.