microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
67.11k stars 3.69k forks source link

[Feature]: Allow specifying custom export conditions in playwright test config #33684

Open mrginglymus opened 2 days ago

mrginglymus commented 2 days ago

🚀 Feature Request

Add option to specifiy additional custom export conditions, analagous to jest's customExportConditions

Example

No response

Motivation

I currently use custom export specifiers to allow switching between raw and compiled sources within my library. This allows for speedy development against local sources with verification that the build process hasn't broken anything, without having to do import mapping in every different bit of tooling.

Currently, playwright does not appear support custom conditions for its test files in the experimental component testing libraries.

Whilst code under test can have this done, by setting:

export default {
  use: {
    ctViteConfig: {
      resolve: {
        conditions: [...]
      }
    }
  }
}

ie, the vite config, no such equivalent exists for the test files themselves.

I have tried launching playwright test with the NODE_OPTIONS env var set to -C <my_condition>; this however prevents yarn from resolving the playwright cli entrypoint:

Error: Cannot find module '...\AppData\Local\Yarn\Berry\cache\playwright-npm-1.49.0-d57440d7c1-10c0.zip\node_modules\playwright\cli.js'
dgozman commented 2 days ago

@mrginglymus Playwright defers to Node.js to resolve export conditions from the package.json. Could you please be more specific and provide an example of what you'd like the API to look like, and how it would import one file instead of another?

mrginglymus commented 1 day ago

Sure! Whilst I can see that playwright is deferring to nodes resolution, there is no way to configure that like there is in vite, jest, webpack, typescript, esbuild and so on.

I've raised this issue in part because the two standard workarounds don't work - setting NODE_OPTIONS="-C <condition>" seems to break the import of lib, and registering a customization hook to inject the condition causes playwright to spin - I'm still investigating what it's doing.

The custom hook, at simplest, looks like this:

export const resolve= async (specifier, context, nextResolve) => {
  return nextResolve(specifier, {
    ...context,
    conditions: ["<condition>", ...context.conditions],
  });
};

and was imported in playwright-ct.config.ts:

import { register } from "node:module";

register("./hook.js", import.meta.url);

What did work was patching this line:

https://github.com/microsoft/playwright/blob/1afb56ee1b63ffbb972a1773432979b865878072/packages/playwright/src/transform/esmLoader.ts#L33

to be

  const result = await defaultResolve(specifier, {...context, conditions: ['<condition>', ...context.conditions]}, defaultResolve);

So my ideal API would be an option somewhere on the playwright test config that allows me to specify a list of additional conditions which are used in any import resolution - or, it could even sniff them from the vite config, to save having to specify them twice.

dgozman commented 1 day ago

@mrginglymus Thanks for the explanation! Looking at our code, I think that passing -C condition should work.

In fact, when I run with NODE_OPTIONS="-C development" npx playwright test, my export condition development is honored in the following package.json:

  "exports": {
    "development": "./dev.js",
    "default": "./def.js"
  },

I wonder how is my setup different from yours. Could you please share a sample repro that does not work?

mrginglymus commented 1 day ago

I suspect the answer is "yarn pnp", but I'll try and put together a repro tomorrow - thanks.

mrginglymus commented 1 day ago

The NODE_OPTIONS issue is actually a combo of yarn pnp and cross-env - still no idea on the hook spin

https://github.com/mrginglymus/pw-eg

Here's a repro - test-ct works, running with cross-env to set NODE_OPTIONS (test-ct-ce) fails with the lib.js import error and running with the hook (test-ct-hook) just spins.

If you switch the nodeLinker to node-modules then the cross env option works; the hook still spins.

The fun part is that if I set NODE_OPTIONS="-C xxx" in my environment directly then it does work.

However!

I wouldn't focus too much on why NODE_OPTIONS isn't working - setting custom export conditions via NODE_OPTIONS is a really poor option as it's a very blunt and brittle tool, as is doesn't stack (ie, one script using NODE_OPTIONS to set a value calling another script also setting NODE_OPTIONS).

dgozman commented 22 hours ago

@mrginglymus Thank you!

It seems like there is some issue with yarn pnp and passing NODE_OPTIONS - perhaps it overrides this env and messes up your -C xxx. Using NODE_OPTIONS is a nice workaround until we figure out whether and how to expose custom conditions, that's why I am trying to push for it.

Playwright registers its own hook, as you've noticed, so when you are registering a second one from the config, we probably trigger https://github.com/nodejs/node/issues/50948. I was able to reproduce this in my setup as well. I guess we'll have to wait for the node issue to be fixed.


Now that we've checked all the options, I'll leave this issue up for prioritization. For now, I'd recommend to make the NODE_OPTIONS approach work, either by option out from pnp or by setting it outside of package.json script as you've mentioned.

mrginglymus commented 21 hours ago

Ah, nice find, that looks like it indeed. Thanks for all the help.