lunarmodules / busted

Elegant Lua unit testing.
https://lunarmodules.github.io/busted/
MIT License
1.4k stars 185 forks source link

Mocking out io standard library breaks the busted #598

Closed mkpankov closed 5 years ago

mkpankov commented 5 years ago

Test:

insulate(
    "should isolate the env",
    function()
        _G.io = nil
        it(
            "should work",
            function()
                -- io.write("Hello")
            end
        )
    end
)

Output:

busted -v _spec_insulate.lua

1 success / 0 failures / 1 error / 0 pending : 0.000562 seconds

Error → _spec_insulate.lua @ 5
should isolate the env should work
.../lua/share/lua/5.2/busted/outputHandlers/utfTerminal.lua:134: attempt to index global 'io' (a nil value)

stack traceback:
        .../lua/share/lua/5.2/busted/outputHandlers/utfTerminal.lua:134: in function 'fn'
        /home/mkpankov/.ciri/lua/share/lua/5.2/mediator.lua:103: in function </home/mkpankov/.ciri/lua/share/lua/5.2/mediator.lua:96>
        (...tail calls...)
        /home/mkpankov/.ciri/lua/share/lua/5.2/busted/core.lua:221: in function </home/mkpankov/.ciri/lua/share/lua/5.2/busted/core.lua:220>
Tieske commented 5 years ago

this is almost impossible to do. Busted uses a lot of 3rd party libraries, and these do not all cache the global in local variables. And then this cannot be prevented. (unless I;'m missing something of course ...)

The solution would be to replace the global (or setting it to nil in your example) within the 'it' block, with a finally clause to revert it.

Something like this:

        it("should work",
            local old_io, _G.io = _G.io, nil
            finally(function() _G.io = old_io end)

            function()
                -- io.write("Hello")
            end
        )

But even then it might fail between a failing test, and before running the finally clause. But at least that's after a test failure to begin with.

mkpankov commented 5 years ago

Thanks for your reply & suggestion. I tried it and my main concern is that with TDD failing tests is norm, and having failures induce errors is too much burden to track.

Maybe you have advice on best practice? I.e. I'd like to mock io.open and io.lines which are used in function I'm testing. I know I can write out my complete mock implementations. What bothers me is that I have to basically reimplement io.lines in my code as it will become more and more complex with tests that demand behavior closer and closer to behavior of real io.lines (i.e. I'd prefer to not bother with format strings).

mkpankov commented 5 years ago

Okay so I went forward and actually tried modifying busted and caching io in every file that uses it. It was surprisingly easy and only required local io = io on top of every file in outputHandlers. Tests pass (aside from one Moonscript one) and my scenario now works w/o a hitch.

Would you be interested in looking at it? I can make a PR.

Tieske commented 5 years ago

A PR would be nice (though I'm not the maintainer of this lib...)

mkpankov commented 5 years ago

@Tieske #599

Tieske commented 5 years ago

this can be closed now.