tarantool / test-run

Tarantool functional testing framework
14 stars 14 forks source link

`--luacov` option doesn't produce `luacov.stats.out` for any kind of tests #419

Open Serpentian opened 5 months ago

Serpentian commented 5 months ago

Steps to reproduce:

  1. Clone vshard and add the following config .luacov to vshard's root directory:
statsfile = 'tmp/luacov.stats.out'
reportfile = 'tmp/luacov.report.out'
exclude = {
    '/test.?',
    '/tmp',
    '/usr',
}
  1. Create tmp dir in vshard. Go to vshard/test and run:
./test-run --luacov --force

All tap tests fail with error:

[003] [test-run server "app_server"] Last 15 lines of the log file /tmp/t/003_unit-tap/scheduler.test.lua.tarantool.log:
[003] LuajitError: /usr/local/bin/tarantool:1: unexpected symbol near 'char(127)'
[003] fatal error, exiting the event loop

The majority of all other tests passes without errors. tmp directory is empty after all tests end, no coverage result.


Details ``` Tarantool 3.1.0-entrypoint-51-g47940311e Target: Linux-x86_64-Debug Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=TRUE Compiler: GNU-13.2.1 C_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/serpentian/Programming/tnt/tarantool=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -fno-gnu89-inline -Wno-cast-function-type -Werror -g -ggdb -O0 CXX_FLAGS: -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/serpentian/Programming/tnt/tarantool=. -std=c++11 -Wall -Wextra -Wno-invalid-offsetof -Wno-gnu-alignof-expression -Wno-cast-function-type -Werror -g -ggdb -O0 ``` `luacov` was installed with: ``` tt rocks install --global luacov ``` Checked `./var`, `/tmp` dirs. Global `find -name ...`, coverage results are not in another directory, `.luacov` is properly read.

test-run adds -l luacov to tarantool executable in order to produce coverage results, for some reason this doesn't work. test-run doesn't use --coverage option of luatest.

Workarounds:

  1. For luatest force --coverage option. This properly produces luacov results for luatest tests:
diff --git a/lib/luatest_server.py b/lib/luatest_server.py
index 0793601..b91d983 100644
--- a/lib/luatest_server.py
+++ b/lib/luatest_server.py
@@ -47,7 +47,7 @@ class LuatestTest(Test):
         # Add luatest as the script.
         command.extend([server.luatest])
         # Add luatest command-line options.
-        command.extend(['-c', '--no-clean', '--verbose', script, '--output', 'tap'])
+        command.extend(['-c', '--coverage', '--no-clean', '--verbose', script, '--output', 'tap'])
         if Options().args.pattern:
             for p in Options().args.pattern:
                 command.extend(['--pattern', p])
./test-run --force luatest
  1. Tap tests can be executed directly by tarantool, without test-run
tarantool -l luacov ./test/unit-tap/scheduler.test.lua
  1. Don't see any way to run diff tests without test-run. No coverage for them at all(