Open Electroid opened 1 year ago
I'm seeing TypeError: jest.mock is not a function as well as TypeError: jest.resetAllMocks is not a function in bun@1.0.25, however, both of those implementations seem to have been checked off the list? Is this a bug?
I'm seeing TypeError: jest.mock is not a function as well as TypeError: jest.resetAllMocks is not a function in bun@1.0.25, however, both of those implementations seem to have been checked off the list? Is this a bug?
jest & related functions are not currently a global. If bun:test is is imported, they have to also be imported. We only auto import expect, describe, test, it right now
I'm seeing TypeError: jest.mock is not a function as well as TypeError: jest.resetAllMocks is not a function in bun@1.0.25, however, both of those implementations seem to have been checked off the list? Is this a bug?
jest & related functions are not currently a global. If bun:test is is imported, they have to also be imported. We only auto import expect, describe, test, it right now
Thank you @Jarred-Sumner, I suspected something like that. I think a lot of us have sadly littered our setup scripts with default mock behavior due to convenience factors. I would like to preserve as much of that as possible while still preserving the bun globals override as it is very convenient.
My example is roughly like
// preload.ts
import bun from 'bun:test'
bun.mock.module("module", () => {
... return some mock
})
// bun:test is imported here
jest.mock("other_module") // TypeError: undefined is not a function (near '...jest.mock...')
describe("test", () => {
...
})
It seem that the above example escapes the override behavior that I want to preserve, so my question is, can I somehow manually tell bun to override it again? Alternatively, is there a source file with the override mappings that I can look at and just replicate it by assigning it to the global object myself? Is there an equivalent for jest.mock('module') that creates automocks? TS keeps complaining that we need to pass the factory argument to module.mock, but I'm not sure if that is really required?
I think this is one of the last blockers of having bun work for some of our tests over at Sentry. Based off some initial tests which are already passing, it seem to be executing them anywhere in the range of 50-2000% faster, which is nothing short of amazing to say the least :)
This would be a workaround, though I am not sure why the mock value is persisted if accessed through window or global, but not when it is being overridden directly
// preload.ts
// jest.mock in test file does not work
Object.defineProperty(jest, 'mock', {
value: bun.mock.module,
writable: true,
});
// global.jest.mock in test file now works
Object.defineProperty(global.jest, 'mock', {
value: bun.mock.module,
writable: true,
});
// window.jest.mock in test file now works
Object.defineProperty(window.jest, 'mock', {
value: bun.mock.module,
writable: true,
});
Is this an appropriate place to ask about the difference between bun test
and jest --runInBand
?
I've noticed some different behavior between these two commands. The Bun docs say:
The test runner runs all tests in a single process.
Which reads a lot like Jest's --runInBand
flag, which says:
Run all tests serially in the current process, rather than creating a worker pool of child processes that run tests. This can be useful for debugging.
But I have experienced different, incompatible behaviors between these two tools. I'm trying to convert the mobx-state-tree tests to Bun, and some of our tests have order-of-operation dependencies, where they expect certain internal cache numbers to have been incremented in a specific order.
I was seeing yarn jest --runInBand
work, but bun test
fail. This diff fixed the issue, where I have written a helper function to reset my action IDs.
The long term solution for me is to make my tests more robust, so they don't rely on internal implementation. Still, I really would have expected bun test
to function like jest --runInBand
, based on the two sets of documentation. Is that the intention and should be fixed, or is it worth it to call out the difference in Bun docs?
That didn't quite fix everything. I have this other very strange behavior that works fine with Jest, but breaks in bun test
.
At this commit, in this file, on this test case: https://github.com/mobxjs/mobx-state-tree/blob/ca0bdaf972d405ea9c5a2a7f42d108adf75f137b/__tests__/core/model.test.ts#L196 (it should show a friendly message
), the expect/toThrow
block breaks a test in a different file.
Here's a video of the behavior. I would do a minimal reproduction, but it's unclear to me what's at play and how I would isolate the problem: https://www.dropbox.com/scl/fi/kdlgqfe4ohh7yjja8bizl/Screen-Recording-2024-02-23-at-4.32.55-PM.mov?rlkey=v6u0ddqeorbm7zb65zer882yu&dl=0
@coolsoftwaretyler I'm having to do this find . -wholename './tests/*_test*' -exec bun test {} \;
@JonCanning - thanks! That makes sense for now - just running each test in a separate process. That may unblock my work for now, but I do think Bun needs to clarify this behavior, because right now it's not a drop-in replacement for -i
(although it's quite close, with better test design, haha).
I really really would love to have test.concurrently
work with bun test, but after scanning the implementation of the test runner it looks like it has some big assumptions on running tests sequentially, for instance this in expect.zig
pub var is_expecting_assertions: bool = false;
pub var is_expecting_assertions_count: bool = false;
seems to be using global state to handle some functionality, which is written to when calling expect
and then read when a test completes in the runner.
Are there any plans to rework this in the future? I feel that async tests would greatly benefit from a concurrent-enabled runner. In the future a parallel-enabled runner would even benefit sync tests.
@dylan-conway, @Electroid There seems to be a disconnect on what is implemented between your docs and this issue tracker. For example, .toHaveBeenLastCalledWith
in your docs is tagged as implemented, but in this issue tracker it's tagged that it still needs to be done. I think it would be worthwhile to do another pass on what is completed so there's no confusion.
@riezahughes we will add support for
expect.extend
and get@testing-library/jest-dom
to work
@Jarred-Sumner Any updates on this one? I was looking at toBeInTheDocument
but it doesn't seem to be implemented yet.
@riezahughes we will add support for
expect.extend
and get@testing-library/jest-dom
to work@Jarred-Sumner Any updates on this one? I was looking at
toBeInTheDocument
but it doesn't seem to be implemented yet.
We added expect.extend
to bun:test
some time ago. toBeInTheDocument
would be implemented within happy-dom, not in bun:test
itself.
happy-dom
@Jarred-Sumner Thanks for the super fast reply! I saw that toBeInTheDocument
is available in happy-dom and read through the docs here: https://bun.sh/docs/test/dom. However, I can't get it to work so that it uses happy-dom as the expect
function is coming from bun:test
which does not have the toBeInTheDocument
method.
Here's an example that says "Property 'toBeInTheDocument' does not exist on type 'Matchers
/// <reference lib="dom" />
import { describe, it, expect } from "bun:test";
import { render, screen } from "@testing-library/react";
import { AdminContext, RecordContextProvider } from "react-admin";
import { MobileField } from "./MobileField";
describe("MobileField", () => {
const unformattedMobile = "+41791234567";
const formattedMobile = "+41 79 123 45 67";
it("formats IBAN correctly", () => {
const data = {
clientProfile: {
mobile: unformattedMobile,
},
};
render(
<AdminContext>
<RecordContextProvider value={data}>
<MobileField source="clientProfile.mobile" />
</RecordContextProvider>
</AdminContext>
);
expect(screen.getByText(formattedMobile)).toBeInTheDocument();
expect(screen.queryByText(unformattedMobile)).not.toBeInTheDocument();
});
});
What's the missing piece for the setup to use happy-dom expect methods together with buns expect
?
@sebastianbuechler The toBeInTheDocument
matcher you're referring to is probably from @testing-library/jest-dom
, not happy-dom
. jest-dom
added Bun support shortly after Bun added support for expect.extend
.
@sebastianbuechler The
toBeInTheDocument
matcher you're referring to is probably from@testing-library/jest-dom
, nothappy-dom
.jest-dom
added Bun support shortly after Bun added support forexpect.extend
.
@jakeboone02 Thanks for pointing this out. You're of course right, it's from @testing-library/jest-dom
.
So just to be clear, if I want to use toBeInTheDocument
expectation I just need to add this code in the test files (or the test setup file):
import * as matchers from "@testing-library/jest-dom/matchers"
export.extend(matchers)
@sebastianbuechler Yes, that should work. TypeScript shouldn't complain either, but if it does then file an issue with jest-dom
.
bun test
is just so fast.
The same test.
bun test src/lang/syntax
:
Executed in 340.83 millis fish external
usr time 396.73 millis 293.00 micros 396.43 millis
sys time 75.11 millis 0.00 micros 75.11 millis
bunx vitest run src/lang/syntax
:
Executed in 5.56 secs fish external
usr time 27.54 secs 0.00 micros 27.54 secs
sys time 3.49 secs 404.00 micros 3.49 secs
But toMatchInlineSnapshot
is the only missing API that blocks me from using bun test
.
Is it much harder to implement than toMatchSnapshot
?
jest.setTimeout is missing as well. If it is not needed then please explain how to make tests work both with node+jest and bun? Is there some global constant exists to check if this is jest or bun environment?
Is test context in the scope? That's so useful!
@sebastianbuechler did you manage to make it work without Typescript complaining? To me is working after exending jest-dom matchers, but seems not to extend the type and Typescript outputs: Property 'toBeInTheDocument' does not exist on type 'Matchers<HTMLElement>'
@fjpedrosa this should be reported in the jest-dom
repo
@sebastianbuechler did you manage to make it work without Typescript complaining? To me is working after exending jest-dom matchers, but seems not to extend the type and Typescript outputs:
Property 'toBeInTheDocument' does not exist on type 'Matchers<HTMLElement>'
@fjpedrosa & @jakeboone02 Just half way. I wanted to define it in a setup function so that I don't have to repeat it, but it seems that typescript does not recognize it that way.
I abandoned bun test as it seems just not mature enough if you have already a lot of tests working with vitest and DOM expectations. Maybe better documentation would help here.
@sebastianbuechler did you manage to make it work without Typescript complaining? To me is working after exending jest-dom matchers, but seems not to extend the type and Typescript outputs:
Property 'toBeInTheDocument' does not exist on type 'Matchers<HTMLElement>'
@fjpedrosa & @jakeboone02 Just half way. I wanted to define it in a setup function so that I don't have to repeat it, but it seems that typescript does not recognize it that way.
I abandoned bun test as it seems just not mature enough if you have already a lot of tests working with vitest and DOM expectations. Maybe better documentation would help here.
I did it. It works. It took me about five hours.
I'm not experienced at configuring stuff, as you may notice by it, but I kept thinking: if someone's smart enough to code the thing I should be at least smart enough to find out how to use it, lol.
Anyway, for anyone who falls here and needs to know how to configure bun with happy-dom and jest-dom with TypeScript and working code completion, that is how I did it in Vite for working with React:
Add dependencies with bun add
:
bun add -D happy-dom @happy-dom/global-registrator @testing-library/jest-dom @types/web
Add this key to the compilerOptions
in tsconfig.json
file:
"types": [
"bun-types", // add Bun global
"@testing-library/react", // if with react
"@testing-library/jest-dom",
"web"
],
happydom.ts
file inside your src folder with the following (this will be pre-loaded so it works in every test file):import { GlobalRegistrator } from "@happy-dom/global-registrator";
const oldConsole = console;
GlobalRegistrator.register();
window.console = oldConsole;
import * as matchers from "@testing-library/jest-dom/matchers";
import { expect } from "bun:test";
// Extend the expect object with custom matchers
expect.extend(matchers);
It is ugly, I know. It works, so I'm not touching it again so soon.
bunfig.toml
file (or create a file named bunfig.toml
at the root level of the project, next to package.json
):[test]
preload = "./src/happydom.ts"
If you want to place the happydom file somewhere else, just make sure it is being parsed by TypeScript.
.d.ts
extension (jest-dom-types.d.ts
, for example) and this code:import "@testing-library/jest-dom";
import type { TestingLibraryMatchers } from "@testing-library/jest-dom/matchers";
declare module "bun:test" {
interface Matchers<R = void>
extends TestingLibraryMatchers<
typeof expect.stringContaining,
R
> {}
interface AsymmetricMatchers
extends TestingLibraryMatchers {}
}
expect
as you normally would in your test file:import { describe, expect, it } from "bun:test";
@cpt-westphalen Step 6 shouldn't be necessary to do manually. Those types should come out-of-the-box with the library. The same PR that fixed the types for expect.extend
generally also added types for bun:test
.
But again, this is not the place to hash out these issues. Any bugs or gaps in @testing-library/jest-dom
should be filed and discussed in that repo and not this one.
Sorry for flooding here with that. I may replace it with the link to the discussion in jest-dom if you prefer.
thanks @cpt-westphalen, I was able to fix the warning with your solution!
Not sure if I'm missing something, but #5356 is still happening for me on 1.1.3
Anyone here get the actual matchers working but run into an issue where jest matcher utils
seems to not be working? I specifically get the following when failing a test using expect().toBeInTheDocument()
:
TypeError: this.utils.RECEIVED_COLOR is not a function. (In 'this.utils.RECEIVED_COLOR(this.isNot ? errorFound() : errorNotFound())', 'this.utils.RECEIVED_COLOR' is undefined)
Anyone here get the actual matchers working but run into an issue where
jest matcher utils
seems to not be working? I specifically get the following when failing a test usingexpect().toBeInTheDocument()
:TypeError: this.utils.RECEIVED_COLOR is not a function. (In 'this.utils.RECEIVED_COLOR(this.isNot ? errorFound() : errorNotFound())', 'this.utils.RECEIVED_COLOR' is undefined)
I was facing the same issue where getAllByText()
was returning me > 1 items from DOM so used the first item from DOM and .getAllByText()[0]
then ran expect().toBeIntheDocument();
https://github.com/oven-sh/bun/issues/1825#issuecomment-2043838793
This is great thank you! I think this is the perfect place to add these instructions. They are extremely necessary to make bun test work with extra matchers that most people are already using and are migrating to bun test now.
~@cpt-westphalen I got an error while running a test, any ideas?~
Figured out the issue, the error below occurs because I was importing the main library, this is not required when using bun test. i.e. I just had to delete this import statement: import '@testing-library/jest-dom';
bun test v1.1.7 (b0b7db5c)
src\TestFile.test.tsx:
5 | import 'aria-query';
6 | import 'chalk';
7 | import 'lodash/isEqualWith.js';
8 | import 'css.escape';
9 |
10 | expect.extend(extensions);
^
ReferenceError: Can't find variable: expect
at C:\Code\frontend\node_modules\@testing-library\jest-dom\dist\index.mjs:10:1
Anyone here get the actual matchers working but run into an issue where
jest matcher utils
seems to not be working? I specifically get the following when failing a test usingexpect().toBeInTheDocument()
:TypeError: this.utils.RECEIVED_COLOR is not a function. (In 'this.utils.RECEIVED_COLOR(this.isNot ? errorFound() : errorNotFound())', 'this.utils.RECEIVED_COLOR' is undefined)
I was facing the same issue where
getAllByText()
was returning me > 1 items from DOM so used the first item from DOM and.getAllByText()[0]
then ranexpect().toBeIntheDocument();
On my end I was also getting the error 'Multiple elements found' and I added this to my happydom.ts
preload file
beforeEach(() => {
document.body.innerHTML = '';
})
I still run into the same error as @paulleonartcalvo with TypeError: this.utils.RECEIVED_COLOR is not a function.
. Have yet to find how to resolve this one. Have you guys found a way?
But
toMatchInlineSnapshot
is the only missing API that blocks me from usingbun test
.Is it much harder to implement than
toMatchSnapshot
?
For me it is very important to have jest.requireActual()
as well as toMatchInlineSnapshot
.
With these two I guess that I could plan a PoC at work.
Do you have a roadmap / expectations to have these two done?
@paulleonartcalvo @immayurpanchal @hussain-s6 @srosato everyone that has the TypeError: this.utils.RECEIVED_COLOR is not a function
... I think this is a 'bad error', it only happened to me when there were actually a problem with the test, like finding more than one element with the criteria on a getBy*
or finding an element on a .not.toBeInTheDocument
assertion. I've always managed to make the test work after messing around debugging it... do you have a case where you are absolutely sure it should be working but is throwing that error?
@rbwestphalen My tests that fail with this error have no problem passing with jest and jsdom. I will try to dig more into it when I get a chance
Temporary implementation for toHaveBeenCalledBefore
and toHaveBeenCalledAfter
.
(inspired by jest-extended)
It would be great if this could be native! 😊
== testSetup.ts ==
import { expect as bunExpect, mock as originalMock } from 'bun:test';
// Define the type for the original mock function
type OriginalMockFunction = (...args: unknown[]) => unknown;
// Define the type for our extended mock function
type ExtendedMockFunction = OriginalMockFunction & {
callOrder: number[];
originalMock: OriginalMockFunction;
};
// Create an extended mock function
function createMock(): ExtendedMockFunction {
const original = originalMock() as OriginalMockFunction;
const mockFunction = ((...args: unknown[]) => {
mockFunction.callOrder.push(++createMock.callOrderCounter);
return original(...args);
}) as ExtendedMockFunction;
mockFunction.callOrder = [] as number[];
mockFunction.originalMock = original;
return mockFunction;
}
createMock.callOrderCounter = 0;
// Custom matchers
function toHaveBeenCalledBefore(
this: { utils: any },
received: ExtendedMockFunction,
secondMock: ExtendedMockFunction,
) {
const firstCallOrder = received.callOrder[0];
const secondCallOrder = secondMock.callOrder[0];
if (firstCallOrder < secondCallOrder) {
return {
pass: true,
message: () =>
`Expected ${received.originalMock.name} to have been called before ${secondMock.originalMock.name}`,
};
} else {
return {
pass: false,
message: () =>
`Expected ${received.originalMock.name} to have been called before ${secondMock.originalMock.name}, but it was called after`,
};
}
}
function toHaveBeenCalledAfter(
this: { utils: any },
received: ExtendedMockFunction,
firstMock: ExtendedMockFunction,
) {
const firstCallOrder = firstMock.callOrder[0];
const secondCallOrder = received.callOrder[0];
if (secondCallOrder > firstCallOrder) {
return {
pass: true,
message: () =>
`Expected ${received.originalMock.name} to have been called after ${firstMock.originalMock.name}`,
};
} else {
return {
pass: false,
message: () =>
`Expected ${received.originalMock.name} to have been called after ${firstMock.originalMock.name}, but it was called before`,
};
}
}
// Custom matchers interface
interface CustomMatchers<T = unknown, R = void> {
toHaveBeenCalledAfter(firstMock: T): R;
toHaveBeenCalledBefore(secondMock: T): R;
}
// Ensure the custom matchers interface extends the Bun matchers interface with consistent type parameters
declare module 'bun:test' {
interface Matchers<T = unknown, R = void> extends CustomMatchers<T, R> {}
}
// Add custom matchers to expect
const expectWithMatchers = bunExpect as typeof bunExpect & {
extend: (
matchers: Record<
string,
(
this: { utils: any },
...args: any[]
) => { pass: boolean; message: () => string }
>,
) => void;
};
// Add custom matchers to expect
expectWithMatchers.extend({ toHaveBeenCalledBefore, toHaveBeenCalledAfter });
// Override the mock function in bun:test
const bunTest = require('bun:test');
bunTest.mock = createMock;
== order.test.ts ==
import './testSetup';
import { describe, expect, it, mock } from 'bun:test';
describe('Function Call Order Tests', () => {
it('should call initialize before process', () => {
const initialize = mock();
const process = mock();
// Simulate the function calls
initialize();
process();
// Verify the call order
expect(initialize).toHaveBeenCalledBefore(process);
});
it('should call finalize after process', () => {
const process = mock();
const finalize = mock();
// Simulate the function calls
process();
finalize();
// Verify the call order
expect(finalize).toHaveBeenCalledAfter(process);
});
it('should call fetchData before processData', () => {
const fetchData = mock();
const processData = mock();
// Simulate the function calls
fetchData();
processData();
// Verify the call order
expect(fetchData).toHaveBeenCalledBefore(processData);
});
it('should call saveData after processData', () => {
const processData = mock();
const saveData = mock();
// Simulate the function calls
processData();
saveData();
// Verify the call order
expect(saveData).toHaveBeenCalledAfter(processData);
});
it('should call setup before execute and then cleanup', () => {
const setup = mock();
const execute = mock();
const cleanup = mock();
// Simulate the function calls
setup();
execute();
cleanup();
// Verify the call order
expect(setup).toHaveBeenCalledBefore(execute);
expect(execute).toHaveBeenCalledBefore(cleanup);
});
});
== Preload ==
bun test --preload ./setupTests.ts
Preloading should work as well but I only tested the code above with direct import.
toContainAnyKeys
has been implemented, but yet to be ticked in the checkbox above..toBeBetween(startDate, endDate)
has not been implemented too, but it is ticked above.Don't know where/how to report this, but as Bun is focused so much on performance and the docs have statements like "Running 266 React SSR tests faster than Jest can print its version number." I can only imagine I run into some problem/bug.
I was finally able to run tests of an existing project in Bun instead of Jest and Bun takes ~36.1s while Jest needs ~12.8s. It's a bit hard to create a minimal example, but I use happydom and Testing Library. Is this result expected? What are realistic numbers?
@donaldpipowitch That is definitely not expected. Would love to be able to profile your test suite and see what's going on.
If you're unable to send code for us to look at, if you're on a mac you could try:
-profile
build of Bun, e.g. https://github.com/oven-sh/bun/releases/download/bun-v1.1.18/bun-darwin-aarch64-profile.zipxcrun xctrace record --template 'Time Profiler' --launch -- ./bun-profile test
.trace
file on Discord or email jarred@bun.sh
Instruments comes with Xcode. It'll look something like this:
Thanks for the quick response. As this is likely a bug then (or bigger misconfiguration from my side) I'll try to create a minimal test case.
@Jarred-Sumner Stupid question, but could it simply be the case that bun is not parallelizing the tests? I tried to break it down, but it becomes less obvious if you run single tests (for single tests bun is a bit faster).
36.03s user 1.79s system 104% cpu 36.080 total
67.84s user 6.81s system 583% cpu 12.785 total
1465.93ms
1799 ms
bun does run all tests single-threaded, but often the cost of parallelizing tests is so high that it doesn't end up being a performance win to use threads
is it using timers? we haven't implemented fake timers yet, and IIRC, the testing library code does something like wait 500 milliseconds of IRL wall clock time in certain cases when there's no jest fake timers
We don't run jest.useFakeTimers()
in our code base (not sure if there is another way to use timers or if this is a default behavior by now). (I also commented out two usages of setTimeout
, but same result.) Will create the .trace
now as it looks like the minimal repo will not really show the problem.
@Jarred-Sumner , sorry to bother you, I was just curious if you got the .trace
file? (Please don't feel pressured. I just want to check, if it reached you.)
dang... no toBeInTheDocument... what the heck bun...
Hey folks, just a follow up on my comment from a few months ago. One of our contributors figured out the differences between bun test and jest, specifically:
AFAICT, bun test maintains globals across modules (unlike jest), so you have to be careful to reset it.
Which was breaking assumptions we had from running Jest and trying to use bun:test
as a fully drop in replacement.
I think this was mostly consumer error on our end, I don't think Bun needs to change, but I thought it might be helpful to leave some breadcrumbs here in case you're also running into issues swapping bun:test
in for a Jest set up that made assumptions similar to ours.
But with this change, we're finally realizing 20x improvements on test suite speed. Phenomenal. Huge. So worth it.
I was finally able to run tests of an existing project in Bun instead of Jest and Bun takes ~36.1s while Jest needs ~12.8s. It's a bit hard to create a minimal example, but I use happydom and Testing Library. Is this result expected? What are realistic numbers?
Re-tested it with 1.1.22
and get the same result.
To clarify @simylein's request — this is about type checker tests for TypeScript, not typeof
tests: https://vitest.dev/guide/testing-types.html
Haven't seen it added to the master list yet, so here I am raising my hand as well 👋
I'm really missing a expect.poll
as well
https://vitest.dev/api/expect.html#poll
Jest
describe()
test()
Lifecycle hooks
expect()
Mocks
Misc
Vitest
expect()
Mocks
Timers
Misc
jest-extended
expect()