bluebird75 / luaunit

LuaUnit is a popular unit-testing framework for Lua, with an interface typical of xUnit libraries (Python unittest, Junit, NUnit, ...). It supports several output formats (Text, TAP, JUnit, ...) to be used directly or work with Continuous Integration platforms (Jenkins, Maven, ...).
Other
565 stars 136 forks source link

Is there a way to modify the level of the error reported at the time of assertion #129

Closed ymh199478 closed 4 years ago

ymh199478 commented 4 years ago

I am developing a lua framework, and I chose to use luaunit when designing unit testing schemes.

I need to wrapped luaunit, but this will break the hierarchy displayed when the assertion is raised.

Can I have a way to modify the level of error

bluebird75 commented 4 years ago

Do you mean the stack trace displayed along with the error ? It would help me understand better if you can provide an example of what you would like to achieve and what is the current result.

ymh199478 commented 4 years ago

I am wrapped the luaunit similar to the following:

If this is used by developers, an exception will cause the following output:

./src/whale/foundation/testing/test-case.lua:15: expected: 1, actual: 2
stack traceback:
        ./src/whale/foundation/testing/test-case.lua:15: in function 'whale/foundation/testing/test-case.assertEquals'
        ./tests/foundation/application-test.lua:45: in function 'tests/foundation/application-test.testBootingCallbacks'

but, my expected output should be:

./tests/foundation/application-test.lua:45: expected: 1, actual: 2
stack traceback:
        ./tests/foundation/application-test.lua:45: in function 'tests/foundation/application-test.testBootingCallbacks'

I am developing a framework project, so I need to wrapped luaunit. Instead of letting developer directly use luaunit, it is not conducive to future expansion.

I have not found a better solution in luaunit that allows me to adjust the error level.

bluebird75 commented 4 years ago

Thanks that's clear now. I have two ideas to address it, I'll evaluate them and tell you. The ideas are either to strip explicitely the lines you don't want from the stack trace or to adjust the error level.

For stripping the stack trace, you can modify the function stripLuaunitTrace() (around line 338 in luaunit.lua ). The other path is to play with the error level failure(), fail_fmt() and error_fmt().

bluebird75 commented 4 years ago

Ok, after a bit of experimenting, you will have to do both options described :

  1. in failure(), fail_fmt() and error_fmt(), adjust the level of error reporting by +1 like that :
-    error(M.FAILURE_PREFIX .. msg, (level or 1) + 1)
+    error(M.FAILURE_PREFIX .. msg, (level or 1) + 2)

The effect is that the first line reported of where the error occured, before the stack trace will no longer reference your internal framework. This should yield something like :

./tests/foundation/application-test.lua:45: expected: 1, actual: 2
stack traceback:
        ./src/whale/foundation/testing/test-case.lua:15: in function 'whale/foundation/testing/test-case.assertEquals'
        ./tests/foundation/application-test.lua:45: in function 'tests/foundation/application-test.testBootingCallbacks'
  1. As you can see, the stack trace is still too extensive. So you will have to modify the function stripLuaunitTrace() to make that disappear.

By the way, it's better to use the latest code (of today) for this. There were a few minor functions which would not go through failure_fmt() or error_fmt()

bluebird75 commented 4 years ago

That's an interesting need by the way. I'll add official support for this in the next release of luaunit.

ymh199478 commented 4 years ago

Thanks, Direct modification of the luaunit code will make future updates difficult, so i am looking forward to seeing this feature in the next version. ^_^

bluebird75 commented 4 years ago

Hi. This ability is now directly included in luaunit. If you use the latest version, look for the variable STRIP_EXTRA_ENTRIES_IN_STACK_TRACEand set it to 1 to get the desired behavior.

ymh199478 commented 4 years ago

Awesome, thanks.