[Feature] Record network requests and save them as ready-to-use mocks

[haikyuu]

I'm using playwright for integration tests. And I can't have a full environment for testing. So I mock api requests and test against them.

When I saw https://github.com/microsoft/playwright-cli/pull/194 this flow for authentication (which is super helpful, thank you), I thought it would be a great idea to generate code for network requests in a file and use them in specific tests (test by test basis in it("...") or beforeEach depending on the scenario.

In this case, I see it like so:

• Adding an option to the cli to save network requests to a given file (file name) like this npx playwright-cli --save-network-requests=flow_1_requests.json codegen
• Adding an API method in playwright to use a mock file or multiple files. This would allow to overwrite previous mocks if they are defined in two files. playwright.useMockFiles(context, ['path/to/flow_1_requests.json', 'path/to/flow_2_requests.json']). It takes the context and paths to mock files.

Same as https://github.com/microsoft/playwright-cli/issues/196

[JoelEinbinder]

It sounds like you want to serve network requests from a har file, and also a cli command to record a har file.

[haikyuu]

Yes, exactly @JoelEinbinder

Also, I tested recordHar option and it's quite powerful as I can:

• Use the har online viewer to see network requests (just like network tab in the browser)
• Change request content (headers, params, body ...) easily as it's a JSON file.
• recording all requests will allow me to run e2e tests without the need to serve my web app (it's quite easy to serve a web app so it's no big deal).

But there are some limitations:

• The size of the file gets too big when there are static assets (js chunks ...)
• It might be not always desirable to serve static content like html and js files. I might be running a web server that serves my static files and I want to mock only XHR requests to the backend. Or I might want to record and run a predefined set of requests.

I think it would be useful to add a filter option while recording as well as when serving a har file. It can be a callback that takes a request and returns true or false.

[Aaron-Pool]

The (mostly) proprietary testing software testIM (which uses playwright under the hood), provides this feature, and it really is very useful. You basically do a sample run that records what all the responses are, and then every time the test re-runs it automatically serves the responses as recorded there. This makes runs from there on out much faster because network speeds are no longer a bottleneck and, assuming you're not intending to test the api as well as the interface, its really useful.

[ahmad2smile]

Was looking for exact same use case and stumbled on this, would be really nice 👍

[kousenlsn]

Any news on this? I was looking for a way to use the HAS file to mock the requests

[kousenlsn]

Well, I ended up creating a lib if anyone ends up needing this while we don't have an inside solution at Playwright itself.

https://github.com/kousenlsn/playwright-request-mocker

It records any network request (using recordHar), clears up the HAR file to only keep XHR requests (persisting into a new mock.json file), then automatically mocks everything following the mock file for any future runs.

Feel free to contribute or report any issues, it is quite green still.

[lifeart]

@kousenlsn is a way to serve HAR prerecorded 3rd pary domain request?

[kousenlsn]

@lifeart Not yet, but, it would be a good feature to add, I'll see if I can add that on the weekend.

[DavertMik]

Use case: I have a page with some requests/responses to server And I prefer to record the snapshots of requests and use it in tests When server is updated I will re-create snapshots

[kousenlsn]

@lifeart I have updated my lib (playwright-request-mocker), now it can read an existing HAR file also, sorry for the delay.

@DavertMik take a look at what I did, you can do just that now.

[DavertMik]

@kousenlsn thanks, that looks really good.

But my goal is to integrate it into a testing framework @codeceptjs so I prefer to wait for a solution by Playwright team.

[mxschmitt]

Sorry for the ping folks! We currently evaluate this feature and need some input to clarify what's needed here. About which of the following cases is this issue, since a few different feature requests are mentioned?

a) recording the traffic of an application and then using e.g. its API response to in the tests? -> Testing the application against mocked API responses. This could be then enabled per file or test level when writing tests. So you have different of theses e.g. called "network snapshots". It's often called web replay. b) record the API responses of individual requests in a test, to have them cached in further tests to speed up the execution? A common example is e.g. a login endpoint, it gets logged in a single time in the beginning and the further execution is cached but it's still a new isolated context in the end. c) record the traffic of an application and select some endpoints which should get mocked e.g. after the codegen. So similar like a) but more like specific on some requests instead of all or a lot of requests.

More input in what use-case you want to cover is of course welcomed, thanks!

@ahmad2smile, @bunchopunch, @p01, @foloinfo, @gkushang, @terje465, @tomatobrown, @PawelWesolowski, @chuckrector, @DorianMaliszewski

[lifeart]

HI @mxschmitt! Likely a + c

My usecase is to be able to do regression / performance tests on "production" copy, and "production" version have a list of 3rd party domains requests (scripts, api, images, etc), and it really hard to get it mocked without complex AST modifications of codebase.

It will be great to have ability to record all domains requests per page, and be able to run playwright with prerecordered HAR, to get 0 "external requests", because all (local and 3rd party) requests will be served from prerecorded HAR.

Once it will be landed, it will be great to have kinda middleware for it, to modify HAR answer during request or bypass some requests to "real" network.

App we use: https://github.com/TracerBench/tracerbench,

Server setup: customized https://github.com/toutpt/har-express + middleware to run multiple experiments

const experimentsToCheck = [
    {
        name: 'index.html',
        enabled: false,
        paths: {
            '/'() {
                // custom index.html
                return fs.readFileSync(path.join(dirname, 'index.html'), 'utf8');
            },
            'clientlibs.js'(candidate) {
                let js = Buffer.from(candidate.content.text, candidate.content.encoding).toString('utf-8');
                js = js.replace("https://3rd.party.domain/script.js", 'data:application/javascript,console.log(1);');
                candidate.content.text = Buffer.from(js, 'utf-8').toString(candidate.content.encoding);
            },
        }
    },
    {
        name: 'no-theme-css',
        paths: {
            'theme-green.210401.css'() {
                // empty css responce
                return '';
            }
        }
    },
]
for (let experiment of experimentsToCheck) {
    const paths = Object.keys(experiment.paths);
    const handlerName = paths.find((name) => {
        return req.path === name || (!name.includes('/') && req.path.endsWith(name));
    });
    if (handlerName) {
        let result = await experiment.paths[handlerName](candidate);
        if (typeof result === 'string') {
            candidate.content.text = Buffer.from(result, 'utf-8').toString(candidate.content.encoding);
            result = undefined;
        }
    }
}

return res.type(candidate.content.mimeType).send(Buffer.from(candidate.content.text, candidate.content.encoding));

[DorianMaliszewski]

Hi,

My usecase is to allow us to launch our integration tests with the updateSnapshots flag enabled (or this can be change by something like --updateMocks=true), then in the test it call the real API (with XHR calls) and saves responses in a file. Next when I want to rerun the test without updating the mocks and instead of doing real API call, it read the file and return responses. IDK if everybody want the same thing. I already have a custom implementation of this feature which look like this repo https://github.com/kousenlsn/playwright-request-mocker from @kousenlsn but with autostart by using test.use({record: true}) 😄 .

I'm in if you want to develop this ! 👍

[Aaron-Pool]

Another use case that someone may have mentioned, but I didn't see in a quick scan, is that it allows you to test your app against a non-idempotent api call without having to do clean up afterwards. Rather than creating (and then having to clean up) a ton of test data that your tests would create from running, it would only create a single data entry, and then use that saved HAR file for future tests.

Additionally, if you run a test twice, once against a captured HAR and once against a real API call, and it only fails on the real API call, you can more quickly identify where the that the regression is in the API, rather than the UI.

[tomatobrown]

My use case is the same as @DorianMaliszewski

[foloinfo]

I wanted the feature type 'a' .

a) recording the traffic of an application and then using e.g. its API response to in the tests?

I personally like ruby's VCR implementation which allows you to wrap any code block to check & record the extrenal requests and save it as a file, then reuse it for future runs.

ex.

playwright.useRecord('external-request-click', async()=> { // or something
  await page.click('id='button-dosomething-withrequest'')
})

[p01]

Thank you @mxschmitt and Playwright core team 🙏

I have two use cases:

1. Removing external calls for static resources ( images & co ) in Component tests.

The first use case is trivial as it's all static resources, and can be handled with simple routes.

2. Replaying API calls to do E2E / integration tests of a user experience "FairyDust" ( say the feature that takes care of handling @ mentions or showing Profile cards, or ... ) into a bigger app "BigApp" ( think Github, Teams, Twitter, ... )

This one is more complex: We log in into BigApp with a test account. BigApp then bootstraps FairyDust and gives it an auth token. FairyDust only knows how to check the expiration of the auth token, to request a new one from BigApp. The auth token is in all the API calls FairyDust makes over XHR or webSocket. It is needed in the request to gate the API calls on the back end, and in the response to update the expiration or invalidate it.

Since we actually log in into BigApp, FairyDust gets a new auth token for every test run on CI, and a straight mapping of request URL+headers+body to response headers+body will not work out of the box.

...

In both cases, we should be able to:

• Specify URL patterns to record/replay
• Run the tests in "record" mode to record the network traffic
• Have the tests fail in "replay" mode if a test makes any unexpected network call as @Aaron-Pool said.

Unexpected network call will happen when a new feature is added to the code tested with Playwright. In that case the author of the code change will need to run the tests in "record" mode to update the network snapshots.

...

I tried to record and (roughly) prune HAR snapshots myself, but it was already a couple of MB for a very simple FairyDust test inside a BigApp.

[bunchopunch]

Hi, @mxschmitt. I'm late to the party, but thanks for asking and thanks for the work y'all do.

I think my use case echos several of the others. Additionally, I haven't actually started using Playwright yet. I'm assessing moving from Cypress and Selenium, part of which is looking for automatic generation of fixtures/mocks. The HAR format sounded like a potentially practical choice, so I wanted to follow along. I found my way to this issue when looking for a VCR type solution, but don't know that VCR would actually solve my use case.

I'm interested in reusing code/test cases for multiple types of tests where possible: integration, e2e, regression, visual, etc. So, I'd like a way to run some of the same tests against either a live API/environment or mocked against some kind of fixture file(s). This feels like it mirrors some of @p01's comment.

Creating fake responses for everything in an application (rest/graphql APIs, images, scripts, 3rd party, etc) and keeping them up to date is obviously a lot of work, so I'd also like a simple process to generate and update those files when necessary. I don't want humans to end up with doing work to make the captured files usable as fixtures after each record.

If I understand, I think I'm looking for something like a or c.

• A way to record traffic used in the tests, maybe recordHar is already enough or a good start.
  • I suspect recording would need to be tied to running the tests against the live environment regardless, so maybe running the tests with a flag as @DorianMaliszewski suggested.
• A mechanism to indicate that that a test run should target either the integration (live environment) or the saved response(s).
• My team is working against several single page applications that will likely already be running locally. So, I can't shake the feeling that something like the filters @haikyuu mentioned or URL patterns p01 mentioned would be useful for reducing the blast radius of the record operation.

Being able to specify particular files, tests, and requests sounds advantageous, especially when making changes that don't impact the whole application. Maybe the filter/pattern piece already mentioned is the same thing. Maybe targeting a specific spec with the command line and the potential record flag helps there. I don't know that it's core to my use case, though.

I also like the idea @lifeart brought up, being able to alter HAR responses. It sounds like it could be useful for exercising multiple test cases, but that sounds like it may also be outside my initial use case. I also don't quite know how those test cases would fit in/play with the ones leveraging recorded fixtures off the top of my head.

Hopefully that helps, despite me not being totally familiar with Playwright yet.

[orihomie]

Also would be great if the requested feature would work with other runners, not just @playwright/test

[DorianMaliszewski]

Also would be great if the requested feature would work with other runners, not just @playwright/test

@orihomie, I think we need to find something that can work in a first way (with playwright/test or not) and then improve it. The first step of adding a feature is to ...adding it 😄 then, in the future improve compatibility and performance 😅.

[unlikelyzero]

@mxschmitt a+b this would also be great for supporting performance testing. If we could easily export HAR Files to disk, they could be used by load testing tools like k6.

[irharrier2]

@mxschmitt @dgozman is there any plans to implement this in near future?

[pavelfeldman]

This will ship as a part of 1.23. See https://playwright.dev/docs/next/network#record-and-replay-requests

[p01]

Thank you @pavelfeldman and team. Will try this ASAP

The comment in the code example of https://playwright.dev/docs/next/api/class-browser#browser-new-context-option-har says that we'll replay the API requests from the HAR, but the example is missing the , urlFilter: "**/api/**" arg to really match what the behavior described by the comment.

Reading https://playwright.dev/docs/next/api/class-browser#browser-new-context-option-har , it's not entirely clear how to catch requests to a URL that matches the urlFilter but isn't in the HAR. Are we supposed to listen to the requestfailed event and match the urlFilter, to decide wheter to fail the test ? It's OK; if this is the intent of the new context HAR options. It would be nice to explain a little how to catch unmet HAR requests.

[p01]

Thank you for the updated documentations and the routeFromHAR(har[, options]) API 🥳

https://playwright.dev/docs/next/network#replaying-from-har https://playwright.dev/docs/next/api/class-browsercontext#browser-context-route-from-har

Am still unsure how to handle aborted requests for URLs that match the urlFilter but aren't in the HAR itself.

Sorry, I didn't get the chance to try Playwright 1.23 and new APIs yet.

[sebinsua]

@pavelfeldman

The URL filter feature is very useful.

Would it be possible to detect that we are retrying a test and only configure page.routeFromHAR if this is the case?

That way I could run the tests against the real back-end first and then if there were failures run against an earlier set of 'safe' mocks. This would allow us to determine whether a test failure is due to the UI or due to the backend.

Edit: Maybe testInfo.retry might work?

[DorianMaliszewski]

The URL filter feature is very useful.

Would it be possible to detect that we are retrying a test and only configure page.routeFromHAR if this is the case?

That way I could run the tests against the real back-end first and then if there were failures run against an earlier set of 'safe' mocks. This would allow us to determine whether a test failure is due to the UI or due to the backend.

Edit: Maybe testInfo.retry might work?

@sebinsua -> You can create a custom fixture that override page and check if its a retry or not 👍

https://playwright.dev/docs/test-fixtures#overriding-fixtures