[Feature]: Conditionally save HAR recordings

[mskelton]

🚀 Feature Request

This has previously been discussed in #28825

Our company uses HAR recordings extensively for network replay in end-to-end tests so we have predictable network traffic when running our tests. When updating the HAR recordings with actual network traffic, we only want to update the saved HAR file on disk if the test passes. Currently, we have hacks to revert changes to the HARs if the test failed to prevent incomplete HARs from being committed.

Adding a function that accepts the test case and test result to allow conditionally saving the HAR based on test status, or other test information would be very helpful in enabling more network recording workflows like ours.

cc @olivierbeaulieu

Example

await context.routeFromHAR('index.har', {
  url: '**',
  update: true,
  updateContent: 'attach',
  shouldSave: (test: TestCase, result: TestResult) => result.status === 'passed',
})

Motivation

While a full network recording and playback solution is not currently in the scope of Playwright as has been discussed previously (https://github.com/microsoft/playwright/issues/31809#issuecomment-2246685946), this feature would allow creating network recording/playback solutions much easier in user land.

[pavelfeldman]

Adding a function that accepts the test case and test result to allow conditionally saving the HAR

When the test is running, would you like to use the network resources or read them off the old HAR?

[mskelton]

@pavelfeldman The approach we use is setting an environment variable for RECORD=true or RECORD=false. If RECORD=true, we call routeFromHAR with update: true, and save the results to our repo. If RECORD=false, we call routeFromHAR with update: false, and use the saved HAR in our repo to replay all network activity in the tests. During record mode, we don't use the saved HARs at all, since we are using real network resources.

The goal here is that when we are running in RECORD=true mode, we can specify to only save the HAR file when the test passes.

@olivierbeaulieu Has documented this approach quite a bit in this issue: https://github.com/microsoft/playwright/issues/18288#issuecomment-1430514334. We have improved the matching logic much more since that original comment, but that at least provides a little bit of info into how some of the matching works.

[pavelfeldman]

The goal here is that when we are running in RECORD=true mode, we can specify to only save the HAR file when the test passes.

Got it.