testing-library / react-testing-library

🐐 Simple and complete React DOM testing utilities that encourage good testing practices.
https://testing-library.com/react
MIT License
18.92k stars 1.1k forks source link

Tests using `render` sigificantly slow down in React 19 #1342

Closed phryneas closed 1 month ago

phryneas commented 1 month ago

Relevant code or config:

import { renderHook, waitFor } from "@testing-library/react";
import { Suspense, use } from "react";

test("does not timeout", async () => {
  const initialPromise = Promise.resolve("test");
  console.time("executing hook");
  console.time("asserting");
  const { result } = renderHook(
    () => {
      console.timeLog("executing hook");
      const result = use(initialPromise);
      console.log("got result in render", result);
      return result;
    },
    {
      wrapper: ({ children }) => {
        return <Suspense fallback={<div>loading</div>}>{children}</Suspense>;
      },
    }
  );

  await waitFor(() => {
    console.timeLog("asserting");
    console.log(result);
    expect(result.current).toEqual("test");
  });
}); //, 300
/** adding the 300ms timeout above would make this test fail */

What you did:

We found this in the Apollo Client tests, where one file went from executing in 10s with React 18 to 80s in React 19.

What happened:

The test takes 300ms longer than expected (if there were multiple suspenseful renders, it would take 600,900 etc.):

% npm test                                                                                                                                                                                                                                                                                       main

> react-use-bug@0.0.0 test
> jest

  console.time
    executing hook: 5 ms

      at wrapper.children.children (src/slow.test.tsx:10:15)

  console.time
    asserting: 24 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    executing hook: 29 ms

      at wrapper.children.children (src/slow.test.tsx:10:15)

  console.log
    got result in render test

      at wrapper.children.children (src/slow.test.tsx:12:15)

  console.time
    asserting: 75 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    asserting: 127 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    asserting: 179 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    asserting: 231 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    asserting: 284 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    asserting: 326 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: null }

      at src/slow.test.tsx:24:13

  console.time
    asserting: 337 ms

      at src/slow.test.tsx:23:13

  console.log
    { current: 'test' }

      at src/slow.test.tsx:24:13

 PASS  src/slow.test.tsx
  ✓ does not timeout (342 ms)

Reproduction:

https://github.com/phryneas/react-19-reproduction-slow-tests

Problem description:

I can seem to only reproduce this with renderHook so far, I had no success reproducing it with render itself (but I might be missing something).

eps1lon commented 1 month ago

I had no success reproducing it with render itself (but I might be missing something).

What was the approach you took for using render? renderHook is really just

function renderHook(useProvidedHook) {
  let result
  function Component() {
    const renderResult = useProvidedHook()
    React.useEffect(() => { result = renderResult })
  }

  render(<Component />)

  return result
}

I'll try to dig deeper into why we're seeing this slow down but I have very little time to do so these weeks.

phryneas commented 1 month ago

Huh, good call on renderHook, turns out that I could simplify this a lot more:

test("does not timeout", async () => {
  const initialPromise = Promise.resolve("test");
  console.time("executing component render");
  console.time("got past the `use` call");
  console.time("assertion succeeded");

  function Component() {
    console.timeLog("executing component render");
    const renderResult = use(initialPromise);
    console.timeLog("got past the `use` call", renderResult);
    return <div>{renderResult}</div>;
  }

  render(
    <Suspense fallback={<div>loading</div>}>
      <Component />
    </Suspense>
  );

  await waitFor(() => {
    screen.getByText("test");
    console.timeLog("assertion succeeded");
  });
});

now logs

    executing component render: 4 ms

    executing component render: 51 ms

    got past the `use` call: 53 ms test

    assertion succeeded: 340 ms

So the timing here seems very much like throttled updates are also applied in tests.

(I updated the reproduction repo)

eps1lon commented 1 month ago

Note: still repros without act:

+global.IS_REACT_ACT_ENVIRONMENT = false;
+configure({
+  asyncWrapper: (callback) => callback(),
+  unstable_advanceTimersWrapper: (callback) => callback(),
+});

I believe this is just the fallback throttle React has that was introduced in https://github.com/facebook/react/pull/26611.

Since the initial render() call commits the fallback, React will wait for 300ms (was 500ms when added) before showing new content to avoid jank.

You should see the same behavior in production.

phryneas commented 1 month ago

I believe in production it's fine/intended, but in tests this is probably too much of a slowdown? (700% in our case)

eps1lon commented 1 month ago

If this is the behavior in production why would we have a different behavior in tests?

If you don't want to use the real timings in tests, you should be enabling fake timers in your tests. You may need to configure at the jest.config.js level i.e. before modules are imported not dynamically via jest.useFakeTimers().

phryneas commented 1 month ago

If this is the behavior in production why would we have a different behavior in tests?

To save money on CI minutes and developer time spent waiting I guess.

If you don't want to use the real timings in tests, you should be enabling fake timers in your tests. You may need to configure at the jest.config.js level i.e. before modules are imported not dynamically via jest.useFakeTimers().

I know about fake timers, but they're really no silver bullet - especially when they can't just be turned on for only one test.


I'm gonna be honest, after this tweet, I had assumed that React 18 already had this at 500ms, and had some workaround of setting the timeout to a lower number in test environments, so I had assumed that this was a regression.

Do you think the React team would be open to consider something like global.IS_REACT_ACT_ENVIRONMENT, maybe global.REACT_FALLBACK_THROTTLE_MS? I'd be happy to file a PR.

eps1lon commented 1 month ago

I'm gonna be honest, after this tweet, I had assumed that React 18 already had this at 500ms, and had some workaround of setting the timeout to a lower number in test environments, so I had assumed that this was a regression.

It just applies to more scenarios in 19 now. In 18, it only applied to nested boundaries.

Do you think the React team would be open to consider something like global.IS_REACT_ACT_ENVIRONMENT, maybe global.REACT_FALLBACK_THROTTLE_MS?

You should file an issue first explaining the issue you're having to get the community involved. Closing this issue here since it's unrelated to React Testing Library since the tests faithfully show production behavior.