ArtemGovorov
commented
7 years ago Wallaby.js from day one was designed to run a single test if only that test was affected by your code changes. This also promotes the good practice of having your tests orthogonal/isolated from each other. There're hundreds more articles about why tests should be independent from each other and about ugly consequences if they are not. There're even testing libs that deliberately shuffle unit tests randomly to enforce the practice.
Performance consideration aside, the issue with your second and third (and forth) tests is that they are just a part of one test - these 2-4 tests are actually one test, just artificially divided into 3 'it' functions.
If I were you, I would ask myself this question: why these are multiple tests (it-s)?
Depending on the answer, I would consider two solutions.
The first solution is to merge them into a single test, like:
describe('MyComponent', () => {
it('...', () => {
// a
let T = snap(MyComponent, store)
// b
T.dispatch(action1)
T.snap()
// c
T.dispatch(action1)
T.dispatch(action2)
T.snap()
// d
T.dispatch(action1)
T.dispatch(action2)
T.tree().props.onChange()
T.snap()
})
})Comments are doing the same, if not a nicer, job to describe what's going on, there's also less unnecessary code.
One may argue that the test is now not testing one single thing - well, it wasn't doing it when it was in 4 it-s either (where a part of each test was hidden in previous tests). Or that breaking in a middle would stop the lengthy test from running - well, it was the same with the 4 original tests.
The second solution was mentioned by you: to set up a context for each test in beforeEach, if required to have nested describe. This way these will truly be self-sufficient tests, able to run/fail independently. So there're benefits, but the solution has a higher maintenance price (more code to write, even though you could extract common bits into separate functions).
describe('MyComponent', () => {
let T
beforeEach(() => {
T = snap(MyComponent, store)
})
it('a', () => {
// T is truthy
})
it('b', () => {
T.dispatch(action1)
T.snap()
})
describe('when actions dispatched', () => {
beforeEach(() => {
// maybe also an function extracted from test 'b'
T.dispatch(action1)
T.dispatch(action2)
})
it('c', () => {
T.snap()
})
it('d', () => {
T.tree().props.onChange()
T.snap()
})
})
})the emerging idiomatic jest+react testing workflow
I'm quite surprised to hear that something that I have always thought of as anti-pattern, is becoming a wide-spread practice. I don't have a huge react/jest testing experience outside of wallaby.js related stuff, so could you perhaps please share some articles/docs/etc. where the practice is described/discussed/encouraged?
Having said all that, as a tool developers (even though willing to enforce best practices), we can definitely consider adding the setting you're requesting, but I'd really like to hear back from you regarding the whole thing, as I sincerely think this way of writing code may hurt in a longer term: like costly maintenance long interdependent tests, migrating between testing frameworks (for example ava runs tests in parallel by default, so would immediately break), etc.
faceyspacey
Issue description or question
Using Jest, variables in an outer scope, whose values are set in another test become undefined after the test you're editing re-runs.
For example:
In this example
Tis set in a previous typically synchronously/sequentially executed test. In the regular Jest test runner, you can depend on variables set in previous tests. In fact making use of the closure created by thedescribeblock is recommended and idiomatic for such things in the Jest community. But with Wallaby, you can't if you intend on editing a test, which ultimately leads to 2 different unpredictable behaviors with wallaby: once when it first runs all your tests, and another when you actually edit a test, both delivering different results. Re-running the entiredescribeblock would simply (or perhaps not so simply) solve this.So in Wallaby, to make the above call to
console.logwork, I have to edit the outerdescribescope, as a workaround. Also note: editing test 1 also won't giveTa value again, and definitely editing test 2 won't.I'm assuming this probably is a very hard bug/issue, and likely not possible given the async nature of Wallaby. To give importance/weight to this issue--and to counter the only solution being to change my code--basically it's highly useful when snapshotting "alive" react components to be able to snapshot a react component and build a "live" instance, which in this case is contained within
Tand then to be able to assert against state changes in SEPARATE SEQUENTIAL TESTS.beforeEachas a solution is also sub-par, because the idea is that you have components that change states in a sequence, so you would have to re-do the previous steps/actions/state_changes in subsequent tests anyway. That sub-par workflow looks like this basically:When really the last test, for example, should only be this:
So you are left repeating steps you did in previous tests to achieve a state that would even have in this example 'onChange' attached to a component instance (i.e.
onChangeonly appears conditionally for the given state). And perhaps the previous dispatches only create that correct state if dispatched in order.So it's highly useful to be able to think of one group of tests as a sequential set of events. I've built a library called
jest-redux-rendererthat nicely keeps your components alive and reactive with redux (Jest only out-the-box keeps components reactive if you callsetStatewhich isn't even idiomatic anymore). I think the concept of "live" components that continue to reactively update is the secret sauce that makes Jest so great for React.Now, if you're able to reactively update in response to redux state, you're going to be doing a lot of tests against a store or component created in a previous test. You benefit HUGELY from being able to manipulate that component idiomatically by dispatching actions and calling attached event handlers. But there is no "maniuplating" i.e. "updating" in Wallaby while editing tests--everything can only be created, or you can use
beforeEachbut that only gets you 10% of the way. So that's the case for this perhaps not previously typical way of testing. In short, it makes a lot of sense for a component instance to stay alive and reactive across multiple sequential tests.Basically the only solution I can see as an option (and perhaps not enabled by default) is to re-run all tests in a
describegroup, including any code in the outerdescribeblock if any single test changes, not just the individual changed tests. That would 100% solve it. I know performance is one of Wallaby's key selling points, so perhaps optionally being to toggle this on for scenarios where it's important to you and you're editing tests makes this palatable from a performance perspective. But I see it as a must for the emerging idiomatic jest+react testing workflow where there is much to gain if your component instances live across tests.Wallaby.js configuration file
Code editor or IDE name and version
Visual Studio Code v1.8
OS name and version
OSX