microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.04k stars 3.6k forks source link

[Bug]: WebKit/macOS - "no-preference" color scheme falls back to the system's one #32862

Open satelllte opened 5 days ago

satelllte commented 5 days ago

Version

1.47.2

Steps to reproduce

Here's a minimal reproduction test:

import {test, expect} from '@playwright/test';

test('has light theme if no preference', async ({page}) => {
  await page.emulateMedia({colorScheme: 'no-preference'});
  const dark = await page.evaluate(() => window.matchMedia('(prefers-color-scheme:dark)').matches);
  console.debug('dark: ', dark);
  await expect(dark).toEqual(false);
});

Here's real-life example: satelllte/space • tests-e2e/theme.spec.ts

Expected behavior

I believe it's expected to have a consistent color preference in any browser, even if no-preference color scheme is set explicitly for the test.

Actual behavior

If dark theme preference is set in macOS, but the test sets no-preference color scheme, it still falls back to the system and chooses the dark one instead of light.

Additional context

Here's a short log example when you run the test code snippet from above:

[chromium] › theme.spec.ts:3:1 › has light theme if no preference
dark:  false
[firefox] › theme.spec.ts:3:1 › has light theme if no preference
dark:  false
[webkit] › theme.spec.ts:3:1 › has light theme if no preference
dark:  true

Environment

System:
    OS: macOS 15.0
    CPU: (14) arm64 Apple M3 Max
    Memory: 5.82 GB / 36.00 GB
  Binaries:
    Node: 21.7.0 - ~/.nvm/versions/node/v21.7.0/bin/node
    npm: 10.8.1 - ~/.nvm/versions/node/v21.7.0/bin/npm
  npmPackages:
    @playwright/test: 1.47.2 => 1.47.2
Skn0tt commented 3 days ago

What's interesting here is that Safari is the only browser that still supports no-preference: https://developer.mozilla.org/en-US/docs/Web/CSS/@media/prefers-color-scheme#browser_compatibility

Skn0tt commented 3 days ago

Alright, we've looked into it more. no-preference was part of an earlier version of the color schemes CSS spec, but was never implemented broadly. Safari initially added support, but then removed it. I've opened an issue with MDN to update it there.

I'll add a note about this to the Playwright docs.

This also means that what you're seeing is the intended behaviour: Browsers don't know what "no-preference" is, so they fall back to the system setting.

satelllte commented 2 days ago

Thank you! That were interesting findings.