playwright-runner missing feature overview

[mxschmitt]

• [x] allow a --headful CLI parameter to start them headful. Currently the custom CLI args can only be a string.
• [x] allow to use the HEADLESS environment variable as an alias for it. Currently it ends up in max callstack exceed error if you try to redefine them and use the values from the CLI argument. Related to #99?
• [x] allow --browser CLI parameter to define the used browser type. Should work.
• [x] allow the browser be set as an env var. might be related to 2)
• [x] built-in the matrix into the playwright-runner, for device + browser support. Not possible right now afaik.

Action items/tbd:

• I would not define the result of CLI argument as a fixture. Pytest is also not doing that and it seems to be confusing. The alternative would be to expose a fixture called "objects" which contains the options or attach it directly to our "Test" data type.

[pavelfeldman]

This all looks good. Not sure I understand the "CLI arguments as Fixture", they are not the same thing, so what exactly do you have in mind? You mean that we blend them in the params among other fixtures?

[mxschmitt]

fixtures.defineParameter('browserName', 'chromium, firefox, webkit', 'chromium');

fixtures.defineWorkerFixture('browserName', async ({browserName}, test) => {
  console.log(browserName) // this would be the CLI argument.
  const browserType = process.env.BROWSER || browserName;
  await test(browserType);
});

thats how we handle it currently which leads to max callstack exceeds. I thought about the following, that we have a fixture called config or parameters which holds the CLI arguments like Pytest is doing:

def launch_browser(pytestconfig):
    print(pytestconfig.getoption("--headful"))

Instead of a getter the commander option struct might make sense, so they do the flag(--) parsing for us.

[pavelfeldman]

The issue above is that we allow same name to be used both in declareWorkerFixture and declareParameter. I wonder if there is a compile-time check against it.

The idea behind parameters is that they are available in the test metaFunction. That way I can fail/fixme the test for headless when my matrix contains both headless and headful.

[pavelfeldman]

I think this is now obsolete