microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.92k stars 3.67k forks source link

[Feature] Allow mocking third party and built-in modules (for non-E2E tests) #14572

Open pkerschbaum opened 2 years ago

pkerschbaum commented 2 years ago

Feature Request

Playwright Test can also be used as a "generic" test runner to run unit/integration tests (see #14268).
For such tests it is often helpful, if not necessary, to mock modules loaded by CJS require or ESM import statements. This allows to test a module in isolation.

At the moment, Playwright does not have a way to replace modules with mocks when they get loaded. As discussed in #14398, Node.js solutions to mock modules like proxyquire or testdouble cannot be used because Playwright has a custom implementation to load modules (for technical reasons) and therefore these solutions don't work.

It would be great if Playwright Test would have some sort of API to mock CJS/ESM modules, similar to the things testdouble is capable of.
Their documentation shows that there would be some things to think through, for example:

Idea

Example is derived from https://jestjs.io/docs/mock-functions#mocking-modules, API inspired by https://github.com/testdouble/testdouble.js#specifying-a-custom-replacement.

users.ts:

import axios from 'axios';

class Users {
  static all() {
    return axios.get('/users.json').then((resp) => resp.data);
  }
}

export default Users;

my-test.spec.ts:

import { test, mock } from '@playwright/test';

import Users from './users';

test('should fetch users', async ({}) => {
  const fakeUsers = [{ name: 'Bob' }];
  /* mock axios.get (would be great if that works even though ./users was imported already */
  mock.replace('axios', {
    get: mock.func(() => Promise.resolve({ data: fakeUsers })),
  });

  const actualUsers = await Users.all();
  expect(actualUsers).toEqual(fakeUsers);
});
aslushnikov commented 2 years ago

@pkerschbaum We do not prioritize this feature since we focus on e2e tests. Let's wait and collect some upvotes.

arvigeus commented 2 years ago

Not sure if that would work, but maybe manually compiling tests with custom tsconfig.json where modules are mapped to their mocked implementations? See here

Example:

{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "axios": ["./playwright/mocks/axios"]
    }
  }
}
pkerschbaum commented 2 years ago

@arvigeus your idea is interesting!
I think this way you could create "global mocks", still it would not have the same feature set as my proposed idea (e.g. you could not define a separate mock per test).

One note on this: TypeScript does not offer any option to replace path aliases in the compiled output, see https://github.com/microsoft/TypeScript/issues/26722 and the more recent https://github.com/microsoft/TypeScript/issues/45862.
There are different solutions to this problem, my current recommendation is to use github.com/nonara/ts-patch in conjunction with github.com/LeDDGroup/typescript-transform-paths.
Like here: https://github.com/pkerschbaum/react-pdf-document-generator/tree/b32fa3b452cda2c0320607e96c3bc7c71965a399/packages/config.

kyllerss commented 1 year ago

Personally I would find this useful to mock-out some back-end services/logic to simplify test setup. I know there are different ways of going about this, but one way it would help me would be to allow me to largely preserve my authentication stack while only mocking a single function server-side.

jeblackburnvmw commented 1 year ago

Let me understand this correctly. There is no way to mock an Axios call to a remote server at present? This is a dealbreaker. I won't be standing up every remote service I need, or build my test suite to be robust enough to make use of arbitrary results returned by a remote service, when what I really need is to run my client-side test suite. If I understand the state of play correctly we won't be using Playwright at all. Which is sad.

RichiCoder1 commented 1 year ago

Let me understand this correctly. There is no way to mock an Axios call to a remote server at present?

This ask is about mocking abitrary JS modules in the context of being a "generic" unit test runner, not network requests. Mocking network requests works today via standard e2e testing: https://playwright.dev/docs/mock

jeblackburnvmw commented 1 year ago

Let me understand this correctly. There is no way to mock an Axios call to a remote server at present?

This ask is about mocking abitrary JS modules in the context of being a "generic" unit test runner, not network requests. Mocking network requests works today via standard e2e testing: https://playwright.dev/docs/mock

Not if the network requests are made with Axios. Mock them all you like in your test, you'll continue to make real requests. I migrated my app to use fetch instead of Axios as an experiment, and the mocks started working.

RichiCoder1 commented 1 year ago

That's odd 🤔. Mocks work with both fetch and XHR (which is what Axios uses IIRC). Not sure why it'd go around that unless something weird was happening. I'd file a ticket or start a discussion in case there's a legit bug there.

ghost commented 1 year ago

Cypress has this module/function mocking built-in, and it's very useful.

Imagine that we want to do integration/end-to-end testing of a feature that uses feature flags, and we don't want to rely on the network response from the feature flag service, which would return data that's not stable, i.e., data that's not controlled by the test case. The ability to mock the feature flag SDK module function would facilitate testing those features and forcing the feature flags into a repeatable known state.

basarat commented 1 year ago

While I agree that all module mocking can be replaced with making the code more parameter driven (instead of import driven), nonetheless modules are a heavy part of JavaScript ecosystems and module mocking is fundamental to modern Testing frameworks.

I think this is also reflected by the number of 👍🏻 it received :rose: https://github.com/microsoft/playwright/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

simon-alvalabs commented 1 year ago

I also see this useful in E2E tests. For example, we call third-party library that resolves a promise and I would very much be able to mock that.

The api in this case looks like this: import {showZXCConnect} from '@zxc-api/connect'; and then you call it: const resultToken = await showZXCConnect(link)

I should love to be able to mock showZXCConnect so it just returns some dummy token.

binomialstew commented 1 year ago

This would be useful in our visual regression tests. We generate lorem ipsum in some storybook stories which are checked for visual diffs. I'd like to mock this lorem generation method to return a constant string when running playwright tests so it doesn't cause a png comparison failure.

bensampaio commented 1 year ago

How many upvotes is enough for this to be prioritized?

littlemyx commented 1 year ago

It is impossible for now to get access or mock at least chrome.storage.local/sync store, which is being used for storing users' authentication data, thus it would be really useful to be able to mock imports for emulation of its behavior. So I'm totally upvoting this feature request.

mastercoms commented 1 year ago

We're using an Auth0 package for our authentication, and it's very cumbersome to test authenticated states otherwise. It would be very much appreciated if this feature cannot be implemented, an alternative provided in the docs for something to replace this pattern.

akira-toriyama commented 1 year ago

+1

FelipeLahti commented 1 year ago

+1 Any workarounds for this? Trying to stub a simple query param used in the component test here.

phcoliveira commented 1 year ago

+1

I use Playwright to unit test the rules of Firebase Database and Firestore.

joshsny commented 10 months ago

Difficult to use Playwright for serious testing without this feature. Only being able to mock external APIs instead of modules forces us to mock implementation details for SDKs whose API calls we cannot make during testing.

idhard commented 8 months ago

I can't run my e2e tests using Nextjs 14 with the directive "server only" as it fails if any of my services include it.

in Vitest i'm able to mock the module

vi.mock("server-only", () => {
  return {
    // mock server-only module
  };
});

This feature would be really useful.

KhaldiAmer commented 8 months ago

+1 This is very important to avoid mocking a lot of requests. We want to mock auth0 for authentication.

MartinDybdahlM commented 8 months ago

+1

janhesters commented 8 months ago

+1 Any workarounds for this? Trying to stub a simple query param used in the component test here.

You can wrap your hole app with MSW before starting the E2E process.

lukasver commented 8 months ago

+1

HawtinZeng commented 8 months ago

+1

jscul commented 7 months ago

We do not prioritize this feature since we focus on e2e tests. Let's wait and collect some upvotes.

Jun 2, 2022

This is a necessary feature for a testing suite. I'm around 6 hours into working with playwright and already have a use-case where this would be incredibly useful. In fact, it's pretty much a blocker.

Even if you're doing e2e, there are certain functions that you need to overwrite to do e2e effectively.

mastercoms commented 7 months ago

As a possible solution/alternative, there is SafeTest:

It combines Playwright + Jest/Vitest as a usable out-of-the-box unified testing suite, supporting mocks.

CodexVeritas commented 7 months ago

+1

Visible-Radio commented 7 months ago

There's no way to mock an esm module? This is frustrating.

jiyuu-jin commented 6 months ago

Probably switching to Cypress - https://www.cypress.io. They have most of playwright's features with mocking included. They even have out of the box mocking for things like system clock, etc.

It's a shame this isn't a priority.

mastershadow commented 6 months ago

+1

hrmJ commented 5 months ago

Specifically with regards to playwright component tests I found a way to "mock" or at least replace some of our local modules using the following workaround:

in playwright.ct.config.ts I added a block for customizing the vite config that pl component tests use for the build. In that vite config I set up aliases for the files I want to replace:

    ctViteConfig: {
      plugins: [...],
      resolve: {
        mainFields: ["module", "browser"],
        alias: {
          "auth/my-auth-module": path.resolve(__dirname, "./src/auth/my-auth-module.mockversion.ts"),
          "@some-3rd-party-lib": path.resolve(__dirname, "./mocks/my-mock-version-of-the-lib.ts"),
        },
      },
kucher-mx commented 3 months ago
Shperung commented 3 months ago

+1

EugeneYaremenko commented 3 months ago

+1

ZherebcovSergey commented 3 months ago

+1

AndreasHald commented 3 months ago

+1 really needed

walid-mansour-thomann commented 3 months ago

+1

ta32 commented 2 months ago

Mocking is useful even for e2e tests. The example in the link is about game with some randomness. It would be very simple to mock the module is responsible for returning a random board. eg: https://www.youtube.com/watch?v=RAFdYqRO2vI (note the project in the video is abandoned)

Also not every dependency is easy to mock by mocking web requests. You could have web apps that connect to usb devices or other devices. In which case mocking network interactions would require too much implementation knowledge of the third party dependency.