Open gsnedders opened 7 years ago
Just curious, how did you get the numbers?
@mrego a script that I thought I had on gist.github.com somewhere but seemingly not; essentially going through all reftests and reading something like the offsetHeight
of the root element.
It'd be really nice to have it around, I realized that I was using 800x600 (that's what Chromium and WebKit use) instead of 600x600 in #9716.
@mrego https://gist.github.com/gsnedders/6721716f6f90380ac6b76acbdbee91a2; I have no idea whether we'll standardise on 800x600 or 600x600, FWIW.
Hey, thanks for the link. I'll give it a try at some point.
So I thought 600x600 was agreed before, if not what I have seen is that Chromium an WebKit use 800x600, dunno what happens in Firefox or Edge. If everybody uses already 800x600 that could be the right value to standardize.
@mrego Firefox just uses wptrunner so it does use 600x600. I have no memory of what Edge does (@thejohnjansen?).
1- So the definitive standard for tests is (should be?) 600px wide by 600px tall?
Test assumptions now state:
The device has a viewport width of at least 800px.
2- A lot of tests could be retrofitted to comply with 600x600. In some tests, that could be more difficult or a lot more difficult ...
3- Tests that occupy more than 1000 (1 thousand) in width or height should be examined or reconsidered ... :
css/CSS2/abspos/abspos-containing-block-initial-001.xht (588x10110) css/CSS2/bidi-text/line-breaking-bidi-001.xht (3608x588) css/CSS2/bidi-text/line-breaking-bidi-002.xht (3608x588) css/css-backgrounds-3/border-radius-clipping.html (160008x160048) css/css-flexbox-1/flex-direction-with-element-insert.html (588x1243) css/css-multicol-1/multicol-fill-auto-block-children-001.xht (1208x588) css/css-multicol-1/multicol-inherit-003.xht (1208x588) css/css-multicol-1/multicol-inherit-004.xht (1208x588) css/css-regions-1/counters/extract-list-items-012.html (588x1180) css/css-regions-1/counters/extract-ordered-lists-in-regions-001.html (588x1317) css/css-regions-1/counters/extract-ordered-lists-in-regions-002.html (588x1156) css/css-regions-1/counters/extract-ordered-lists-in-regions-003.html (588x1200) css/css-regions-1/counters/extract-ordered-lists-in-regions-explicit-counters-001.html (588x1501) css/css-regions-1/counters/extract-ordered-lists-in-regions-explicit-counters-003.html (588x1335) css/css-regions-1/counters/extract-ordered-lists-in-regions-explicit-counters-004.html (588x1373) css/css-regions-1/counters/extract-ordered-lists-in-regions-explicit-counters-005.html (588x1392) css/css-regions-1/counters/extract-unordered-lists-in-regions.html (588x1317) css/css-regions-1/writing-modes/extract-ordered-list-direction-and-writing-modes.html (588x1544) css/css-text-3/text-transform/text-transform-capitalize-001.html (588x1636) css/css-text-3/text-transform/text-transform-capitalize-003.html (588x3476) css/css-text-3/text-transform/text-transform-capitalize-005.html (588x1716) css/css-text-3/text-transform/text-transform-capitalize-007.html (588x2516) css/css-text-3/text-transform/text-transform-capitalize-010.html (588x1796) css/css-text-3/text-transform/text-transform-capitalize-011.html (588x1156) css/css-text-3/text-transform/text-transform-capitalize-014.html (588x1876) css/css-text-3/text-transform/text-transform-capitalize-016.html (588x3076) css/css-text-3/text-transform/text-transform-capitalize-018.html (588x3796) css/css-text-3/text-transform/text-transform-fullwidth-001.xht (588x12415)
etc...
This one css/css-writing-modes-3/flexbox_align-items-stretch-writing-modes.html (17895698x588) certainly looks suspicious...
@TalbotG
So the definitive standard for tests is (should be?) 600px wide by 600px tall?
See #7598; we're not consistent here, though the upstream maintained runners use 600x600. It seems likely that we'll change that to be larger, however.
A lot of tests could be retrofitted to comply with 600x600. In some tests, that could be more difficult or a lot more difficult ...
That's definitely true. That said, there's definitely an argument as to whether it's a worthwhile use of resources to fix these tests versus other things people could work on, especially if we do change the (default) size.
Tests that occupy more than 1000 (1 thousand) in width or height should be examined or reconsidered ...
From a brief glance, they're mostly either deliberately doing that or ex-Presto tests (which were 1600x1200).
The documentation states variously that reftests are taken at 600x600 and 800x600; they're mostly run at 600x600 and this apparently matches what was decided for the CSS testsuite years ago so let's just go with that.
Note we also have a
<meta name='viewport-size'>
that's totally undocumented and that only works for Servo per w3c/wptrunner#166.This follows on from #4619 but provides a far better list of affected tests and includes the now merged CSS testsuite.