Open peabnuts123 opened 7 years ago
I closed the other issue to continue discussion here.
My current PR #654 has a commit at the start that fixes a bug actually unrelated to the PR but more related to this issue. It came up in the discussion on #535. Obviously it is nowhere to a complete fix, but it plugs one more small hole.
Unfortunately with out top level language features to facilitate this the amount of deep-copy shenanigans we would have to do to really and truly sandbox this is probably a never-ending game of whack-a-mole.
I'm happy to facilitate PRs which improve the state of affairs, but am skeptical that the sieve can be made completely water-tight with any amount of effort. Pragmatically I'd help address real world cases, but don't have time to contribute as a research project fixing theoretical problems.
Invoking a new host/process for each test file would be nice, and/or adding a new variant of describe()
which achieves this (so that a single file could be split across multiple envs/processes). But then handling test output and status becomes more difficult I suppose. The current method of sandboxing is kinda confusing (where is sandboxing occurring? and what kind of sandboxing exactly?) and falls short in sandboxing scope (especially when Lua is embedded and is accessing state that is internal to the process, but external to Lua).
Using the default options, it also seems that the behavior of these 2 differ:
describe('', function()
function Global() end
end)
describe('', function()
function _G.Global() end
end)
The sandboxing concept of "Insulation" in Busted is nice but seems to promise a bit more than it offers. What is sold as a snapshot of your environment that is restored upon exiting an Insulation block boils down to three things
_G
_G
itselfpackage.loaded
This means that there are still ways to leak across the "border" of insulation. One example is through properties of globals within
_G
, whose state is not captured/restored, as demonstrated below:The ramifications of this are that tests can interfere with each other, despite being within
insulate
blocks, or even different files. There should be a more robust way of capturing/restoring the state of the environment, perhaps even running separate instances of Lua to ensure "true" sandboxing.