react: Testing thrown errors

perymimon commented 2 years ago

Describe the feature you'd like:

When I want to test a situation that my tested hook throws an expected error I like I have the option to test it without hacking the test code. It will be greeted if the implementation that actually done on that subject on react-hooks-testing-library will be implemented on @testing-library/react

Suggested implementation:

As describe here result.error should be hold the throwing object and accessing to result.current will should throw the error again

eps1lon commented 2 years ago

We definitely want to document a good pattern for testing errors. But this should cover components as well not just hooks.

perymimon commented 2 years ago

It is nice, but implantation of already made work not should be delayed by feature work on components

ckknight commented 2 years ago

To get the functionality that was previously available in @testing-library/react-hook, I came up with this chunk of code:

(it's basically the same code as renderHook, with the extra error-handling)

import type { RenderHookOptions } from '@testing-library/react';
import { render } from '@testing-library/react';
import * as React from 'react';

function customRenderHook<TProps, TResult>(
  renderCallback: (initialProps: TProps) => TResult,
  options: RenderHookOptions<TProps> = {},
) {
  const { initialProps, wrapper } = options;
  const result = {
    current: undefined as TResult | undefined,
    error: undefined as unknown,
  };

  function TestComponent({
    renderCallbackProps,
  }: {
    renderCallbackProps: TProps | undefined;
  }) {
    let pendingResult: TResult | undefined;
    let pendingError: unknown;
    try {
      pendingResult = renderCallback(renderCallbackProps!);
    } catch (error) {
      pendingError = error;
    }
    React.useEffect(() => {
      result.current = pendingResult;
      result.error = pendingError;
    });

    return null;
  }

  const { rerender: baseRerender, unmount } = render(
    <TestComponent renderCallbackProps={initialProps} />,
    {
      wrapper,
    },
  );

  function rerender(rerenderCallbackProps?: TProps) {
    baseRerender(<TestComponent renderCallbackProps={rerenderCallbackProps} />);
  }

  return {
    result,
    rerender,
    unmount,
  };
}

export { customRenderHook as renderHook };

TadeuA commented 2 years ago

I agree with perymimon, you don't need to wait to have a component error resolution to raise the hooks error resolution.

Now with the arrival of React 18 and the non-update of the lib @testing-library/react-hooks, I would be very grateful if this could be implemented as soon as possible.

eps1lon commented 2 years ago

I agree with perymimon, you don't need to wait to have a component error resolution to raise the hooks error resolution.

The problem is that without understanding the problemspace fully we might create an interface that only works with hooks but not components. If we rush hooks implementation and later realize that it's not a good interface for testing errors in components, we have to either release another breaking change or fragment the testing landscape. Neither of these scenarios is ideal.

We already have issues by letting /react-hooks proliferate. This library encouraged testing practices that are not in line with the Testing Library mentality and caused bigger migration issues when going from 17 and 18. A scenario which we want to avoid and something we wanted to fix by using Testing Library.

To elaborate more on the issues I see with saving the last error:

it's no longer apparent from your test what caused the error (e.g. in "render -> fireEvent -> fireEvent -> assert on result.error" it's not clear what caused the error. Something that might be important).
only the last error is saved. Is only the last error relevant or should you assert on every single one?
what should we do with console.error that React might add as well (e.g. "The above error occured in ..."). Some testing setups throw on console.error calls so we might want to ensure that no spurious console.error goes through when you asserted on a error

From my experience it's probably better to move this to matchers i.e. provide an API along the lines of expect(() => renderHook()).toThrow('some error message').

erosenberg commented 1 year ago

For anyone this may help, I wrote a very simple helper function to catch just an error thrown by a hook. I'm sure it can be improved, but I wanted to write a reusable function that didn't override renderHook but ran it through a try/catch as was suggested in other threads.

import { renderHook } from '@testing-library/react' // v14.0.0

// This suppresses console.error from cluttering the test output.
export const mockConsoleError = () => {
  jest.spyOn(console, 'error').mockImplementation(() => {})
}

export const restoreConsoleError = () => {
  if (console.error.mockRestore !== undefined) {
    console.error.mockRestore()
  }
}

export const catchHookError = (...args) => {
  let error
  mockConsoleError()

  try {
    renderHook(...args)
  } catch (e) {
    error = e
  }

  restoreConsoleError()
  return error
}

Here is an example of it in use:

hooks/useCustomHook.js

export const useCustomHook = (name) => {
  if (!name) {
    throw new Error('Please provide a name')
  }
}

hooks/useCustomHook.test.js

it('should throw an error', () => {
  const error = catchHookError(() => useCustomHook())
  expect(error).toEqual('Please provide a name')
})

joebobmiles commented 1 year ago

@eps1lon So.... we are letting documentation for a feature that doesn't exist yet block the implementation of said feature?

Sounds a little like putting the cart before the horse to, as you put in this comment, "start with a documented pattern first before building any API around it."

While it isn't the end of the world for me that React Testing Library can't handle errors (I have only one test in y-react that relies on this), it is annoying to see this get shoved under the rug.

nanxiaobei commented 2 months ago

import { renderHook } from '@testing-library/react';
import { expect, vi } from 'vitest';

const getHookErr = (hook: () => void) => {
  let err;
  const spy = vi.spyOn(console, 'error').mockImplementation(() => {});
  try {
    renderHook(hook);
  } catch (e) {
    err = (e as { message?: string }).message;
  }
  spy.mockRestore();
  return err;
};

expect(getHookErr(() => useErrHook())).toEqual('error message');

aliechti commented 2 months ago

@erosenberg and @nanxiaobei, thank you for sharing your examples! I encountered a similar issue when testing Suspense-based hooks and wanted to share the solution that worked for me.

Solution

Here’s the utility I created to handle this:

function expectToThrow(fn: () => Promise<unknown>) {
  const consoleErrorSpy = jest.spyOn(console, 'error').mockImplementation();
  return expect(fn().finally(() => consoleErrorSpy.mockRestore()));
}

Example Tests

Below are a few tests demonstrating how this utility works in different scenarios:

import {useEffect} from 'react';
import {renderHook, act} from '@testing-library/react';

function expectToThrow(fn: () => Promise<unknown>) {
  const consoleErrorSpy = jest.spyOn(console, 'error').mockImplementation();
  return expect(fn().finally(() => consoleErrorSpy.mockRestore()));
}

beforeEach(() => {
  jest.useFakeTimers(); // Use fake timers before each test
});

afterEach(() => {
  jest.useRealTimers(); // Restore real timers after each test
});

it('should throw an error immediately', async () => {
  function useErrorHook() {
    throw new Error('Simulated Error');
  }

  expectToThrow(async () => {
    renderHook(() => useErrorHook());
  }).rejects.toThrow('Simulated Error');
});

it('should throw an error inside useEffect', async () => {
  function useErrorHook() {
    useEffect(() => {
      throw new Error('Simulated Error');
    }, []);
  }

  expectToThrow(async () => {
    renderHook(() => useErrorHook());
  }).rejects.toThrow('Simulated Error');
});

it('should throw an error after a delay', async () => {
  const delay = 100;

  function useErrorHook() {
    useEffect(() => {
      const timer = setTimeout(() => {
        throw new Error('Simulated Error');
      }, delay);

      return () => clearTimeout(timer);
    }, []);
  }

  renderHook(() => useErrorHook());
  expectToThrow(async () => {
    jest.advanceTimersByTime(delay);
  }).rejects.toThrow('Simulated Error');
});

it('should throw an error after a suspense delay', async () => {
  const delay = 100;
  let storedError: Error;

  const timerPromise = new Promise<string>((_, reject) => {
    setTimeout(() => {
      const error = new Error('Simulated Error');
      storedError = error;
      reject(error);
    }, delay);
  });

  function useErrorHook() {
    if (storedError) {
      throw storedError;
    }
    throw timerPromise;
  }

  renderHook(() => useErrorHook());
  expectToThrow(async () => {
    await act(async () => {
      jest.advanceTimersByTime(delay);
    });
  }).rejects.toThrow('Simulated Error');
});

it('should not throw an error when no error occurs', async () => {
  function useNoErrorHook() {
    useEffect(() => {
      // No error thrown
    }, []);
  }

  await expectToThrow(async () => {
    renderHook(() => useNoErrorHook());
  }).resolves.not.toThrow();
});

eps1lon commented 1 month ago

You can test errors in Hooks just like you test errors in Components:

Components:

function Thrower() {
  throw new Error('I throw')
}

test('it throws', () => {
  expect(() => render(<Thrower />)).toThrow('I throw')
})

Hooks:

function useThrower() {
  throw new Error('I throw')
}

test('it throws', () => {
  expect(() => renderHook(useThrower)).toThrow('I throw')
})

In React 19 you will see additional console.warn call instead of console.error unless you wrap the state update in act (which render and fireEvent from React Testing Library already do). Then you won't see any additional console calls.

In React 18, you'll see additional console.error calls that you'd need to expect. How you do this depends on your setup. We generally don't recommend just mocking console.error because it may hide legit errors. Instead, you should assert on each warning explicitly.

Docs are added in https://github.com/testing-library/testing-library-docs/pull/1416 (Preview ("How do I test thrown errors in a Component or Hook?"))

There's no need to add any additional API to React Testing Library.

Testing error boundaries will be explained separately since it's not tied to Hooks. I'll do this in the same docs PR for https://github.com/testing-library/react-testing-library/pull/1354.

aliechti commented 1 month ago

@eps1lon what about the async and suspense cases?

eps1lon commented 1 month ago

What cases are those specifically? If you throw during render, it doesn't matter if there's Suspense or not.

errors in async tasks are up to the author. You should test those just like your users would encounter them.

testing-library / testing-library-docs