tarantool / test-run

Tarantool functional testing framework
14 stars 14 forks source link

Enable luatest logging #429

Closed ochaplashkin closed 1 month ago

ochaplashkin commented 2 months ago

tap13: fix parsing non-utf8 chars

The log file may contain non-utf8 characters which will cause an error:

  TAP13 parse failed ('utf-8' codec can't decode byte 0xbe ...

Replace non-utf8 characters when reading a file.

Part of #427

Enable logging to unified file

If the test fails and there is a unified log file(run.log by default) you'll see the following output:

- [box-luatest/varbinary_test.lua, null]
  # logfile:         /tmp/t/log/031_box-luatest.log
  # reproduce file:  /tmp/t/reproduce/031_box-luatest.list.yaml
  # unified logfile: /tmp/t/031_box-luatest/run.log

Closes #427

coveralls commented 2 months ago

Coverage Status

coverage: 62.625%. remained the same when pulling 9727b6c9143bf034483c89cf5b803dfa3b4fada8 on ochaplashkin:bump-luatest-to-add-more-logs into dd0006329d3c9ca30a956b66e8b61db1a57e51b9 on tarantool:master.

ochaplashkin commented 1 month ago
Statistics:
* pass: 1881
* disabled: 73
* skip: 8
* fail: 5
Failed tasks:
- [box-luatest/gh_9797_netbox_on_disconnect_error_hangs_server_test.lua, null]
# logfile:        /tmp/t/log/067_box-luatest.log
# reproduce file: /tmp/t/reproduce/067_box-luatest.list.yaml
# luatest logfile: /tmp/t/067_box-luatest/run.log
- [box-luatest/gh_9309_errors_in_triggers_test.lua, null]
# logfile:        /tmp/t/log/058_box-luatest.log
# reproduce file: /tmp/t/reproduce/058_box-luatest.list.yaml
# luatest logfile: /tmp/t/058_box-luatest/run.log
- [box-luatest/gh_9994_next_prefix_iterator_test.lua, null]
# logfile:        /tmp/t/log/067_box-luatest.log
# reproduce file: /tmp/t/reproduce/067_box-luatest.list.yaml
# luatest logfile: /tmp/t/067_box-luatest/run.log
- [box-luatest/varbinary_test.lua, null]
# logfile:        /tmp/t/log/061_box-luatest.log
# reproduce file: /tmp/t/reproduce/061_box-luatest.list.yaml
# luatest logfile: /tmp/t/061_box-luatest/run.log
- [box-luatest/gh_7904_export_box_schema_version_to_public_api_test.lua, null]
# logfile:        /tmp/t/log/061_box-luatest.log
# reproduce file: /tmp/t/reproduce/061_box-luatest.list.yaml
# luatest logfile: /tmp/t/061_box-luatest/run.log

Patch:

diff --git a/.gitmodules b/.gitmodules
index 229062bea..716b602c8 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -17,7 +17,7 @@
    ignore = dirty
 [submodule "test-run"]
    path = test-run
-   url = https://github.com/tarantool/test-run.git
+   url = https://github.com/ochaplashkin/test-run.git
    ignore = dirty
 [submodule "src/lib/small"]
    path = src/lib/small
diff --git a/test-run b/test-run
index dd0006329..e85db54ee 160000
--- a/test-run
+++ b/test-run
@@ -1 +1 @@
-Subproject commit dd0006329d3c9ca30a956b66e8b61db1a57e51b9
+Subproject commit e85db54eec64bcbd066416f7e1c7a680e31fb6c1
diff --git a/test/box-luatest/gh_7904_export_box_schema_version_to_public_api_test.lua b/test/box-luatest/gh_7904_export_box_schema_version_to_public_api_test.lua
index 18af7968f..9953e2b28 100644
--- a/test/box-luatest/gh_7904_export_box_schema_version_to_public_api_test.lua
+++ b/test/box-luatest/gh_7904_export_box_schema_version_to_public_api_test.lua
@@ -23,7 +23,7 @@ g.test_box_internal_schema_version_deprecation = function(cg)
     local deprecation_warning =
         'box.internal.schema_version will be removed, please use box.info.schema_version instead'
     t.assert_is_not(cg.server:grep_log(deprecation_warning, 256), nil)
-    local log_file = g.server:exec(function() return box.cfg.log end)
+    local log_file = g.server:exec(function() return rawget(_G, 'box_cfg_log_file') or box.cfg.log end)
     fio.truncate(log_file)
     cg.server:exec(function()
         box.internal.schema_version()
diff --git a/test/box-luatest/gh_9309_errors_in_triggers_test.lua b/test/box-luatest/gh_9309_errors_in_triggers_test.lua
index 5c142adbf..d636f3708 100644
--- a/test/box-luatest/gh_9309_errors_in_triggers_test.lua
+++ b/test/box-luatest/gh_9309_errors_in_triggers_test.lua
@@ -95,7 +95,9 @@ g.test_ctl_triggers_error = function()
         box.ctl.on_shutdown(function() error("on_shutdown error") end, nil)
     end)

-    local server_log_path = g.server:exec(function() return box.cfg.log end)
+    local server_log_path = g.server:exec(function()
+        return rawget(_G, 'box_cfg_log_file') or box.cfg.log
+    end)
     g.server:drop()
     t.assert(g.server:grep_log("on_shutdown error", nil,
         {filename = server_log_path}))
diff --git a/test/box-luatest/gh_9797_netbox_on_disconnect_error_hangs_server_test.lua b/test/box-luatest/gh_9797_netbox_on_disconnect_error_hangs_server_test.lua
index bff0cbc37..727c7dbbd 100644
--- a/test/box-luatest/gh_9797_netbox_on_disconnect_error_hangs_server_test.lua
+++ b/test/box-luatest/gh_9797_netbox_on_disconnect_error_hangs_server_test.lua
@@ -24,7 +24,7 @@ g.test_on_disconnect_error_hangs_server = function(cg)
     local log_file = cg.server:exec(function()
         box.ctl.set_on_shutdown_timeout(1)
         -- `grep_log` will not be able to retrieve it after we drop the server.
-        return box.cfg.log
+        return rawget(_G, 'box_cfg_log_file') or box.cfg.log
     end)
     cg.server:drop()
     t.assert_not(cg.server:grep_log('on_shutdown triggers failed', 1024,

Failed tests: box-luatest/varbinary_test.lua and box-luatest/gh_9994_next_prefix_iterator_test.lua with following error:

TAP13 parse failed ('utf-8' codec can't decode byte 0xbe in position 4198: invalid start byte)
ochaplashkin commented 1 month ago

Done

Statistics:
* pass: 1886 (flaky: 1)
* disabled: 73
* skip: 8