search

web-platform-tests / interop

web-platform-tests Interop project
https://wpt.fyi/interop
280 stars 28 forks source link

Form Controls #11

Closed mfreed7 closed 2 years ago

mfreed7 commented 2 years ago

Description Form controls are mentioned frequently as a source of developer frustration. While "form controls" is a quite general area, there are several recurring themes (e.g. see the most recent state of css survey results):

Specification Other than accent-color, form element appearance is not (well) specified.

Tests

Rationale Forms and forms-related-interop problems are very common developer complaints, e.g. state of CSS.

jgraham commented 2 years ago

So I'm in favour of improving form controls in general, but I'm concerned about the lack of spec/tests for the areas that are causing developer pain. Is there an area that meets the bar of:

If not it seems like the first step in this area would be to agree on the path forward at the specification level and revisit including this in a future iteration of the Interop initative.

jensimmons commented 2 years ago

@mfreed7 What is your proposal for inclusion of a specific web technology to be included in Interop 2022?

"Improve form controls" is the mission of the Open UI project, and the HTML and CSS working groups. Interop 2022 is for encouraging browser rendering engines to implement features that have specifications, or to fix interop bugs that are particularly problematic for web developers.

bkardell commented 2 years ago

It strikes me that it might be easier for me to explain my comments/questions in the meeting and on the other issue with specific examples and questions.

"form element stylability" is too general thing probably, and is not well defined enough and seems mostly wound up in openui sutff. There might be something very specific worth looking at there, but I think it would need to be specific. However, unprefixed appearance:none/auto seems like it could be a specific example of something I would include in this bucket, and seems appropriate for interop 2022. There are a number of existing tests (linked above) there that have plenty of gaps we could close, and there might be more. Form submission and validation behaviors too have lots of tests and are very ragged, and maybe there should be more. input has many gaps in https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input#browser_compatibility. The example on #9 could go in a similar bucket for forms or interactive elemnts compat.

Inevitably, for some of these things there will be reasons like "the spec is unclear" or "the spec left this undefined, but there is a defacto standard, it seems".

This part has potential overlap with other venues, but I believe that's ok - not so much solving a new problem as filling observable gaps?

There are things, including PRs with tests like https://github.com/whatwg/html/issues/5099 which point out things like this - is that 'in scope'? If so, I would say that also could go in a "forms" (or "interactive elements") bucket.

I think there are deltas on datalist too, but there are few tests. On twitter someone also suggested that there are "cross-browser differences in which events fire when. E.g., while typing each character in an input or keyboard scrolling through a select list." Not sure if those are known or have tests, but again, seems to fit a possible 'theme' here and if so would fit into what I'd expect would probably be appropriate.

zcorpan commented 2 years ago

For appearance (at least for the behavior of the property, not necessarily the precise rendering for all widgets), spec and test PRs are in review and I hope can be merged soon:

mfreed7 commented 2 years ago

Thanks for the comments here. I think I may have confused things by starting my list with "Form element stylability". I didn't intend this issue to be about forward-looking things that OpenUI is working on. As several have pointed out, that type of "new stuff" likely doesn't belong here.

But I do think perhaps some corner case form control stylability things could be included here? Things that provide developer frustration currently? Due to the poor state of form element standardization, perhaps many of these are going to end up being controversial. But just for a quick example, <input type=week value='2021-W45'> shouldn't match the appearance of <input type=text value='2021-W45'>. I.e. there should be a display that represents the week. This meets maybe 2.5 of the three bars:

I also just want to really +1 the comment from @bkardell. It seems like there's a collection of behaviors that do meet the bars above. It'll take some work to collect them, but that's the intention of this issue. @josepharhar from the Chromium team is willing to try to do that collection, if folks think this is a worthwhile area to include in Interop2022.

jensimmons commented 2 years ago

perhaps some corner case form control stylability things could be included here? Things that provide developer frustration currently?

Rather than spending more time debating whether or not we should perhaps consider a to-be-determined-in-the-future list of possibilities, I’d like to assert that yes, this is an area folks are willing to consider. There’s no one objecting to suggestions that concern form controls.

What’s needed is a specific list. What exactly is proposed? As we make decisions about what will be included in Interop 2022, we will be considering specific technology or bugs.

Without a list, we won’t be able to make any decisions. We can’t just “write a blank check”.

mfreed7 commented 2 years ago

Rather than spending more time debating whether or not we should perhaps consider a to-be-determined-in-the-future list of possibilities, I’d like to assert that yes, this is an area folks are willing to consider. There’s no one objecting to suggestions that concern form controls.

Great! I guess I misunderstood your first post. As I said, @josepharhar will work on putting together a proposed list.

josepharhar commented 2 years ago

Here is a concrete list of things we could do, with specific actions bolded.

Other thoughts:

Note: When I say "entries" or "spreadsheet", I'm talking about the rows of this state of CSS feedback spreadsheet: https://docs.google.com/spreadsheets/d/1W1scqBvz6xRERPGOCm_29cc71U9CJiIlfeNNyys-27M/edit#gid=1399297592

gsnedders commented 2 years ago

<input type=range>

  • The pseudo elements aren't specced, aren't tested, and aren't in every browser. We could spec and test these pseudo elements.

To be clear, I presume you’re suggesting we (browser vendors collectively) do this work in the appropriate group (be in the WHATWG HTML workstream, the W3C CSS WG, or W3C Open UI CG?

On the whole, I think including anything which doesn’t already have a specification (in at least draft form) should probably be excluded, as otherwise we’re aiming to go from a blank page to an interoperably implemented specification within a single year, which seems very ambitious at best, and utterly unrealistic at worst.

foolip commented 2 years ago

This issue could probably be handled similarly to the suggestion in https://github.com/web-platform-tests/interop-2022/issues/4#issuecomment-969106336, including the well-spec'd and -tested things, and some more future-looking work happening, but not as part of a metric.

bkardell commented 2 years ago

Given the confusion in the description and discussion here - just for ease of following, can I suggest that we open a new issue with the specifics and close this one... it's too easy to start at the top and immediately read it differently than I think several of us are talking about which does not include future looking work. I don't think editing the OP is better.

josepharhar commented 2 years ago

It seems like maybe I should be providing a list of existing WPTs instead of suggesting spec or test changes...?

jgraham commented 2 years ago

Based on the confusion here, I think a new issue with a list of:

will be needed to assess the proposal for Interop 2022.

If there are additional areas where you think we aren't there yet, but where it might be possible to make enough progress on pre-implementation work (e.g. specs) in the next year to include in a future iteration of this scheme, open an additional issue for those parts, explaining the situation.

jensimmons commented 2 years ago

maybe I should be providing a list of existing WPTs instead of suggesting spec[…] changes?

That’s right.

This group does not have the ability to work on specifications. Web standards are created by working groups (and community groups) that have a rigorous process for ensuring the ideas generated by the group are dedicated to open source and can become part of the web. We cannot to put together an ad-hoc group to invent something new, without the legal contracts in place to protect the ownership of that invention.

For example, in order to be a member of the CSS Working Group, you have to either be an Invited Expert who signs a stack of documents pledging the legal ownership of the work you do to the W3C, or you are an employee of a corporation, to which you've pledged ownership of the work you are doing to them, and they've formally joined the W3C and agreed that the work there done by their employees is owned by the W3C. It's a big deal, and involves a lot of lawyers.

This group is not a working group for writing specifications. We are simply communicating about our separate efforts to implement specs that were created elsewhere, to share tests to see if we've done a good job with those implementations, and to make it easy to talk about these efforts with web developers.

This group can collaborate on manual testing and writing up a report of findings (as suggested to better understand the current state of viewport measurements #4). Such findings could then be taken up by a Working Group as a to-do list for spec work. But there's no need for Interop 2022 to do so for the topic of form controls, because the Open UI Community Group was already formed to do exactly that. It does not make sense to duplicate efforts in two places.

If you want to help make headway improving interop for form controls as part of Interop 2022, please refer to existing web standards (with links), and highlight areas where browsers do not match those standards (with tests). Opening a new issue with a much tighter scope is a good idea.

The larger question of improving developer experience of styling form controls is especially tricky, because the web standards for form controls are vague and leave a lot up to interpretation by the browser. Form controls are not 100% interoperable because of the long history of them being considered part of the browser UI or operating system UI, not part of a web page. Each browser decides for themselves how they want their color picker or calendar widget to look & work. And that’s not going to change anytime soon. Such controls are designed to be easily understood by the end user, in the context of using their device.

If you are interested in tackling the larger conundrum — figuring out how to make it possible for web developers to more easily create branded form controls consistent across browsers, despite the fact each browser will decide for itself how to make default controls — then I do suggest you get involved with Open UI. I know progress is slow, and perhaps you want things to happen much more quickly, but this is an incredibly complex space. It will take years to solve these needs, and Open UI is already making great progress.

josepharhar commented 2 years ago

As @mfreed7 mentioned, we've put together a specific list of form element interop areas, each with a spec, tests, and developer interest. All of these are "backwards looking" - they cover areas that are already standardized and have tests written. We would like to propose this list be used as the "Form Controls" item for Interop-2022.

Here is the proposed list:

jensimmons commented 2 years ago

Thanks for creating this list, @josepharhar. This makes discussing form controls very concrete. Excellent.

For WebKit bug tickets…

gsnedders commented 2 years ago

input elements, especially type=month and type=week

Looking at the data from chromestatus.com:

type usage
hidden 48.53514%
text 45.42959%
submit 25.57569%
search 16.79710%
checkbox 11.37675%
button 6.47834%
email 5.17911%
radio 4.55302%
password 4.32131%
file 3.45936%
number 2.28061%
range 2.27417%
image 1.35899%
tel 1.08187%
url 0.26108%
date 0.19424%
reset 0.13829%
time 0.01665%
color 0.00930%
datetime-local 0.00632%
month 0.00141%
week 0.00012%

AFAICT, the reasons why month/week are so comparatively low aren't because of lack of interop, but because they are relatively niche. (And indeed, part of the reason why Safari on macOS lacks support for them is because macOS doesn't provide month or week pickers either.) Focusing on them especially seems perhaps unjustified, given I don't see much evidence of developer interest? It is, after all, easy to use the native ones then fallback.

The survey results linked do cite date/time inputs, but I don't think we have any clear signals as to what the problems are. Given those survey results are focused on CSS, one might assume it's about styling the inputs, which those tests don't cover (and isn't specified anywhere).

There's certainly interop issues there—I'm not questioning that—but it's the question of whether it's a priority, especially with any focus we want for Interop 2022.

At least personally, I think the rest of the things in the above comment (i.e., aside from input elements) are not unreasonable, but I do wonder if we should split them into smaller chunks than a singular "form controls" one.

josepharhar commented 2 years ago

Thanks for the feedback!

appearance — actually, this is the ticket: https://bugs.webkit.org/show_bug.cgi?id=143842. Could you update the link in your comment?

Thanks, I added this bug to the appearance section of my comment. I kept the new webkit bug I filed in there as well though because 143842 is marked as fixed and doesn't appear at first to be about the new spec behavior that will hopefully be changed soon.

input elements, especially type=month and type=week — could we break this out into two items

Done! I filed two new WebKit bugs for the the-input-element directory of WPTs and included them in my list.

Then let's ignore https://bugs.webkit.org/show_bug.cgi?id=200416 as a duplicate of work that's already been completed & shipped

This bug is still open and includes week and month in its description... should I try to close that one and open new one(s) that are just for week and month?

annevk commented 2 years ago

Sam makes a pretty compelling case to exclude week and month controls. I also couldn't find anything in the CSS survey that specifically relates to those.

SebastianZ commented 2 years ago

Regarding events on disabled form controls: https://bugzilla.mozilla.org/show_bug.cgi?id=1220048

Sebastian

foolip commented 2 years ago

I have now labeled (almost) all tests listed in https://github.com/web-platform-tests/interop-2022/issues/11#issuecomment-976047408 with the interop-2022-forms label: https://wpt.fyi/results/?label=master&label=experimental&product=chrome&product=firefox&product=safari&aligned&q=label%3Ainterop-2022-forms

@josepharhar can you go through that and see if I seem to have missed anything? I excluded crash and tentative tests, and also the tests from https://github.com/web-platform-tests/wpt/pull/30267 are still in flight.

As we review this test list, looking at tests that don't pass or fail in the usual manner is worthwhile, mostly timeouts and harness errors: https://wpt.fyi/results/?label=master&label=experimental&product=chrome&product=firefox&product=safari&aligned&q=label%3Ainterop-2022-forms%20%28status%3A%21ok%26status%3A%21pass%26status%3A%21fail%29

Most likely some of the tests could be improved to fail faster and with a clearer reason.

josepharhar commented 2 years ago

I have now labeled (almost) all tests listed in #11 (comment) with the interop-2022-forms label:

I don't see the interop-2022-forms label on these:

josepharhar commented 2 years ago

As we review this test list, looking at tests that don't pass or fail in the usual manner is worthwhile, mostly timeouts and harness errors: https://wpt.fyi/results/?label=master&label=experimental&product=chrome&product=firefox&product=safari&aligned&q=label%3Ainterop-2022-forms%20%28status%3A%21ok%26status%3A%21pass%26status%3A%21fail%29

@domenic It looks like these tests you added are timing out on safari. I think history.back() is causing safari restart the test instead of navigate the iframe...? https://wpt.fyi/results/html/browsers/browsing-the-web/navigating-across-documents/replace-before-load?label=master&label=experimental&product=chrome&product=firefox&product=safari&aligned&q=label%3Ainterop-2022-forms%20%28status%3A%21ok%26status%3A%21pass%26status%3A%21fail%29 http://wpt.live/html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/form-submit.html

domenic commented 2 years ago

Yeah, timeouts are a pretty common failure case for navigation-related tests. I believe I've filed a Safari bug on those.

josepharhar commented 2 years ago

Yeah, timeouts are a pretty common failure case for navigation-related tests

If there is a way to make them fail without timing out, that would be awesome. I tried to do it myself but it seems like the test is rapidly restarting and I couldn't figure out how to detect the case where history.back() was used to start the test. If not, I guess that's OK as long as they are still included in interop2022.

I believe I've filed a Safari bug on those.

Can you find the bug? It would be great to link it to it on wpt.fyi

domenic commented 2 years ago

https://bugs.webkit.org/show_bug.cgi?id=227299

foolip commented 2 years ago

In https://github.com/web-platform-tests/wpt-metadata/pull/2401 @ziransun is suggested that we drop a number of tests that involve week/month. They are:

https://wpt.fyi/results/html/semantics/forms/constraints/form-validation-checkValidity.html https://wpt.fyi/results/html/semantics/forms/constraints/form-validation-reportValidity.html https://wpt.fyi/results/html/semantics/forms/constraints/form-validation-validity-stepMismatch.html https://wpt.fyi/results/html/semantics/forms/the-input-element/input-valueasdate-stepping.html https://wpt.fyi/results/html/semantics/forms/the-input-element/input-valueasnumber.html https://wpt.fyi/results/html/semantics/forms/the-input-element/input-valueasnumber-stepping.html https://wpt.fyi/results/html/semantics/forms/the-input-element/input-valueasdate.html

I've gone through them, and for all of them the subtests all pass in Chrome and Firefox, and the only failures in Safari are related to week/month. That makes it a pretty clear case for dropping the tests, so I'll merge that PR to unlabel the tests. (If there had been a bunch of other subtests that we did want, we'd need to split the tests.)

foolip commented 2 years ago

A number of tests for this area came up in my analysis in https://github.com/web-platform-tests/interop-2022/issues/48. Quoting:

/dom/events/Event-dispatch-click.html is TIMEOUT in Chrome, but that's the expected failure mode when the event doesn't fire.

The 6 form-submit* tests in /html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/ are ERROR in Safari. There's a NoSuchFrameException raised in Python so probably there's some WebDriver issue underneath this.

The 2 form-requestsubmit-during-* tests in the same directory are TIMEOUT in Safari. Needs investigation.

5 form-validation-* tests in /html/semantics/forms/constraints/ have MISSING tests because two different tests are created (and fail) in Safari than in Chrome and Firefox. These tests are about week/month and like web-platform-tests/wpt-metadata#2401 should be excluded, but the tests need to be fixed to run the same subtests in all browsers first to see if there are other failing tests that aren't about week/month.

/html/semantics/forms/constraints/infinite_backtracking.html is TIMEOUT in Firefox, presumably because it has the bug the test is probing for.

/html/semantics/forms/form-submission-0/form-double-submit-multiple-targets.html is TIMEOUT in Safari.

/html/semantics/forms/form-submission-0/form-double-submit-to-different-origin-frame.html is TIMEOUT in Firefox and Safari, but also fails in Chrome, tracked by https://crbug.com/1087077. I would guess one of the events never fire in Firefox/Safari and that the timeout is legit.

The rel-* tests in /html/semantics/forms/form-submission-target/ are TIMEOUT in Safari, but all of the subtests pass. This very unusual and needs investigation.

/html/semantics/forms/textfieldselection/select-event.html is TIMEOUT in Safari after the first subtest times out. Looks like the "select" event never fires and the test is OK.

/html/semantics/forms/textfieldselection/selection-not-application.html has fewer subtests in Safari, the week/month ones aren't created. This actually seems fine from a metrics point of view if we score each browser individually.

/html/semantics/forms/textfieldselection/selection-start-end.html is an overall TIMEOUT in Safari, with a bunch of NOTRUN subtests. What's odd is that no subtest has timed out. Needs investigation.

/html/semantics/forms/textfieldselection/textfieldselection-setRangeText.html is an overall TIMEOUT in Safari after one of the subtests is TIMEOUT due to an event not being fired. The test seems OK, but it's not obvious how to score this.

/html/semantics/forms/textfieldselection/textfieldselection-setSelectionRange.html is TIMEOUT in Safari because one subtest is TIMEOUT. Also tricky for scoring.

/html/semantics/forms/the-input-element/range-restore-oninput-onchange-event.html is TIMEOUT in Safari, both harness and subtests. An event isn't fired, test seems OK.

I would encourage folks to search for the name of their browser in this comment and see if anything needs investigation/fixing.

A bunch of the issues are about week/month input. https://github.com/web-platform-tests/wpt-metadata/pull/2401 didn't remove all of those tests, but for what remains tests will have to be split. @josepharhar is this something you might look into? (I didn't check who wrote the tests.)

josepharhar commented 2 years ago

https://bugs.webkit.org/show_bug.cgi?id=227299

Thanks! I linked this bug here: https://github.com/web-platform-tests/wpt-metadata/pull/2409

I've gone through them, and for all of them the subtests all pass in Chrome and Firefox, and the only failures in Safari are related to week/month. That makes it a pretty clear case for dropping the tests, so I'll merge that PR to unlabel the tests. (If there had been a bunch of other subtests that we did want, we'd need to split the tests.)

Sounds good to me

The 6 form-submit* tests in /html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/ are ERROR in Safari. There's a NoSuchFrameException raised in Python so probably there's some WebDriver issue underneath this.

Oh, I didn't notice this, I thought they were just timeout. I now see that they say "MISSING". When you open the test on wpt.live, the page constantly reloads very fast in safari, maybe that's why there are other issues? As opposed to an actual infra problem?

/html/semantics/forms/textfieldselection/selection-start-end.html is an overall TIMEOUT in Safari, with a bunch of NOTRUN subtests. What's odd is that no subtest has timed out. Needs investigation.

I'll try fixing this test if I can.

/html/semantics/forms/textfieldselection/selection-not-application.html has fewer subtests in Safari, the week/month ones aren't created. This actually seems fine from a metrics point of view if we score each browser individually.

A bunch of the issues are about week/month input. web-platform-tests/wpt-metadata#2401 didn't remove all of those tests, but for what remains tests will have to be split. @josepharhar is this something you might look into? (I didn't check who wrote the tests.)

Isn't selection-not-applicable.html the only one? I'll try splitting it into two tests.

josepharhar commented 2 years ago

I just wrestled with selection-start-end.html for a while, and I'm happy to just let it keep timing out until webkit implements the corresponding behavior.

I made a PR to separate week and month cases from selection-not-application.html: https://github.com/web-platform-tests/wpt/pull/32479

foolip commented 2 years ago

The 6 form-submit* tests in /html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/ are ERROR in Safari. There's a NoSuchFrameException raised in Python so probably there's some WebDriver issue underneath this.

Oh, I didn't notice this, I thought they were just timeout. I now see that they say "MISSING". When you open the test on wpt.live, the page constantly reloads very fast in safari, maybe that's why there are other issues? As opposed to an actual infra problem?

Taking form-submit-button-click.html as an example, the test is indeed MISSING, but above that you'll see "Harness status" and "ERROR". If you toggle "Show Details" you'll see this text squashed into a narrow box:

ERROR message: Traceback (most recent call last):
  File "/Users/runner/work/1/s/tools/wptrunner/wptrunner/executors/executorwebdriver.py", line 406, in run_func
    self.result = True, self.func(self.protocol, self.url, self.timeout)
  File "/Users/runner/work/1/s/tools/wptrunner/wptrunner/executors/executorwebdriver.py", line 496, in do_testharness
    result = protocol.base.execute_script(
  File "/Users/runner/work/1/s/tools/wptrunner/wptrunner/executors/executorwebdriver.py", line 50, in execute_script
    return method(script)
  File "/Users/runner/work/1/s/tools/webdriver/webdriver/client.py", line 20, in inner
    return func(self, *args, **kwargs)
  File "/Users/runner/work/1/s/tools/webdriver/webdriver/client.py", line 791, in execute_async_script
    return self.send_session_command("POST", "execute/async", body)
  File "/Users/runner/work/1/s/tools/webdriver/webdriver/client.py", line 659, in send_session_command
    return self.send_command(method, url, body, timeout)
  File "/Users/runner/work/1/s/tools/webdriver/webdriver/client.py", line 623, in send_command
    raise err
webdriver.error.NoSuchFrameException: no such frame (404):

Opening http://wpt.live/html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/form-submit-button-click.html in STP 137 indeed just keeps reloading. I assume the problem is that the form is submitted and reloads the page. When wptrunner then tries to poll the page for test results (I think) the original page is gone.

Is there any way for this test to fail before the navigation happens, or is the navigation happening the bug itself that the test is supposed to detect? If it is, then I think we'll have to live with the ERROR, that it's a surprising manifestation of a real bug.

A bunch of the issues are about week/month input. web-platform-tests/wpt-metadata#2401 didn't remove all of those tests, but for what remains tests will have to be split. @josepharhar is this something you might look into? (I didn't check who wrote the tests.)

Isn't selection-not-applicable.html the only one? I'll try splitting it into two tests.

Thanks for splitting! There are also week/month subtests in these tests:

josepharhar commented 2 years ago

Is there any way for this test to fail before the navigation happens, or is the navigation happening the bug itself that the test is supposed to detect? If it is, then I think we'll have to live with the ERROR, that it's a surprising manifestation of a real bug.

I tried querying window.history to see if the main page was navigated backwards to, which I think is the case where the test keeps restarting, but I couldn't quite figure it out.

Thanks for splitting! There are also week/month subtests in these tests:

https://github.com/web-platform-tests/wpt/pull/32499

jensimmons commented 2 years ago

For reference, this is what we considered and decided:

Technology considered Decision
appearance Include
accent-color Investigate
<form rel> Include
Events on disabled form controls Include
input type=month + type=week Exclude
Input elements bugs Include
Form submission bugs Include
Form validation bugs Include