Suggestions for 'prefersColorScheme' custom metric

[demianrenzulli]

Thanks again @tomayac for working on this script.

I started to analyze how sites have implemented a dark theme and decided to use the custom metric to see if worked. The sites I'm currently analyzing are: https://mobile.twitter.com, https://web.whatsapp.com, https://www.github.com, https://www.youtube.com and https://pwa.zdf.com.

Based on what I could see there, I'm suggesting the following changes:

• Allowing for zero (instead of one) or more spaces between @media and the parenthesis.
• Including a regex for window.matchMedia() as some sites seem to be checking prefers-color-scheme via JS.
• Checking all the response bodies, instead of just the stylesheets, as developers tend to add the dark CSS themes directly in the HTML or JS.

This last one is controversial, and I guess you might have already thought about it. I recall you mentioned that libraries like Facebook use it for tracking purposes. There will be also other false positives, like Web.dev (test), which doesn't provide dark mode, but contains articles that talk about how to implement it. In any case, it seems like the false negatives for only checking CSS files could be higher. If you disagree, we can go back to the previous one.

Another thing we can do is to create a separate metric: prefersColorSchemeLax, which parses all the files. Then we can compare the delta, and if there are some libraries like Facebook, we can crate an exclusion list.

Additional tests:

• Twitter Test: Used to return false and now returns true. They use the prefers-color-scheme directly in the HTML.
• Github Test. Used to return @media(prefers-color-scheme: dark) because they have no space between @media and the parenthesis: @media(prefers-color-scheme: dark). Now returns true.
• Youtube Test uses window.matchMedia(), so it returns true.

I hope this helps.

cc // @andreban @una @beverloo @rviscomi

[demianrenzulli]

Thanks for working on this @tomayac! I was thinking that matching only stylesheets and scripts would leave out cases like Twitter, that has inlined in the HTML the portion of the CSS code that provides the dark functionality.

I know inlining is not so uncommon, and it's a technique we recommend in some cases to improve render time, but I can't have an idea of how frequently developers do this.

Please, let me know your thoughts.

Also, thanks for improving the regular expression!

[tomayac]

Pages that inline the CSS like Twitter would typically still contain JavaScript that controls the experience, concretely https://abs.twimg.com/responsive-web/client-web/main.34d0caa5.js#:~:text=matchMedia(%22(prefers-color-scheme:%20dark) at time of this writing. I'm a little worried about the signal becoming useless if it triggers for blog posts like web.dev/prefers-color-scheme/ about but not using the feature.

[demianrenzulli]

It sounds good to me. I had another idea to complement this research (by using another tool). I'll open a thread for that, but for the time being, it seems like this is good.

[demianrenzulli]

By the way, I applied the suggestions, but I don't have permissions to merge. I guess you might have, so, if you think it's fine, please, go ahead.

Thanks for reviewing this PR!

[tomayac]

Likewise thanks for working on this. Merged!