search

lunarmodules / busted

Elegant Lua unit testing.
https://lunarmodules.github.io/busted/
MIT License
1.4k stars 185 forks source link

TAP output invalid if print function overridden #632

Closed Luca-spopo closed 3 years ago

Luca-spopo commented 4 years ago

In my modules, I override the _G.print function to contain extra information such as line numbers. I imagine this is a common scenario. The insulate blocks correctly reset the print function after a test is complete, but any TAP output produced during the test becomes invalid as it uses the overriden function. (This issue may also be present in other output handlers)

The fix is very simple. We can cache the value of the print function in TAP.lua before any test is executed.

Luca-spopo commented 4 years ago

Example output: We can see here that test points 24 to 27 are fine, but then 28 to 30 have test cases which set _G.print to a custom function, which invalidates the output Invoked using -v --defer-print -o tap


# ../tests/baton_spec.lua @ 136: lib.baton.lua active device check - note: Baton does not currently change active device for just mouse inputs
ok 25 - lib.baton.lua active device check - note: Baton does not currently change active device for just mouse inputs
# ../tests/baton_spec.lua @ 230: lib.baton.lua supports dummy table
ok 26 - lib.baton.lua supports dummy table
# ../tests/baton_spec.lua @ 178: lib.baton.lua released and pressed callbacks
ok 27 - lib.baton.lua released and pressed callbacks
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 49 : "# ../tests/root_spec.lua @ 14: Exposing fakelove Checking conf.lua is valid"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 60 : "ok 28 - Exposing fakelove Checking conf.lua is valid"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 49 : "# ../tests/root_spec.lua @ 8: Exposing fakelove fakelove raises an error if we print an error"
Tue Jun 23 23:00:56 2020 ../tests/root_spec.lua 9 : "there was an ERROR"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 35 : "not ok 29 - Exposing fakelove fakelove raises an error if we print an error"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 36 : "# ../tests/root_spec.lua @ 8"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 37 : "# Random seed: 882730804"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 38 : "# Failure message: ../tests/root_spec.lua:9: Expected a different error.\n# Caught:\n# (no error)\n# Expected:\n# (error)"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 40 : "# stack traceback:\n# \t../tests/root_spec.lua:9: in function <../tests/root_spec.lua:8>\n# "
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 35 : "not ok 30 - Exposing fakelove Checking main.lua is valid Loading main.lua with GS=common.MainMenu"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 36 : "# ../tests/root_spec.lua @ 21"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 37 : "# Random seed: 882730804"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 38 : "# Failure message: ../tests/root_spec.lua:22: attempt to index field 'isDown' (a function value)"
Tue Jun 23 23:00:56 2020 ../rocks/share/lua/5.1/busted/outputHandlers/tap.lua 40 : "# stack traceback:\n# \t../tests/root_spec.lua:22: in function <../tests/root_spec.lua:21>\n# "```
Luca-spopo commented 4 years ago

Looking at other files, I think TAP should be using io.write (io is already cached) instead of print. Need an expert to confirm if that's the right thing to do.

Tieske commented 4 years ago

that was my thinking as well, using io.write 👍