w3c / webdriver-bidi

Bidirectional WebDriver protocol for browser automation
https://w3c.github.io/webdriver-bidi/
351 stars 38 forks source link

Add command to check that an Element is visible #530

Open whimboo opened 11 months ago

whimboo commented 11 months ago

Checking elements for visibility is hard and in WebDriver classic we make use of a Selenium atom. We should provide a similar command for WebDriver BiDi.

css-meeting-bot commented 11 months ago

The Browser Testing and Tools Working Group just discussed Support for "Is Element visible".

The full IRC log of that discussion <jgraham> Topic: Support for "Is Element visible"
<jgraham> github: https://github.com/w3c/webdriver-bidi/issues/530
<sadym> q+
<sadym> q-
<orkon> ScribeNick: orkon
<shs96c> q+ for even more entertaining things people want to do with visibility
<orkon> jgraham: test authors want to figure out if the element is possible to see if my app is working
<orkon> jgraham: the most obvious thing is to check the bounding client rect is on screen
<orkon> jgraham: and then turns out the visible means white one white, transforms etc
<cb> bromann
<orkon> jgraham: we discussed it in Classic endlessly but ended up putting the code into the appendix based on Selenium
<orkon> jgraham: so the question to what extend we want to revisit this to bidi?
<orkon> q+
<orkon> jgraham: we know have preload scripts and not it is easy to send the scripts to the page and we wanted to update the javascript sometimes
<orkon> jgraham: we have seen regressions when we updated Atoms
<orkon> jgraham: there are possibilities 1) we do nothing, i.e., use the preload script to detect that whenever you want 2) we spec the same set of scripts 3) revisit the whole thing
<orkon> jgraham: 3) (smth not based on the script)
<orkon> ack next
<Zakim> shs96c, you wanted to discuss even more entertaining things people want to do with visibility
<orkon> shs96c: there was one thing you missed. In WD Classic we use the visibility atom to check if the text is displayed because there was no innerText. It is still more consistent in many cases.
<orkon> shs96c: I think a lot of ability to check if the element is visible might be only in the render tree
<orkon> shs96c: I am not sure we will be able to define it well in w3c spec
<gsnedders> q+
<orkon> shs96c: my option would be to go with option 1
<orkon> ack next
<sadym> q+
<shs96c> orkon: just wanted to mention that option 1 sounds good (laughs) but if we want a solution, we should break the definition of "visible" into different parts. In puppeteer, visibility checks are used for figuring out if an element can receive actions (eg. is clickable) Maybe we could split out that part (eg. "what makes an element clickable")
<shs96c> q+
<gsnedders> q-
<shs96c> orkon: we don't need to cover all aspects, but we can figure out which assertions are relevant. We could probably do hit testing on specific clickable points
<orkon> ScribeNick: orkon
<orkon> q?
<orkon> ack next
<jgraham> RRSAgent: make minutes
<RRSAgent> I have made the request to generate https://www.w3.org/2023/09/15-webdriver-minutes.html jgraham
<orkon> sadym (IRC): +1 what alex said. How can preload script be used for visibility?
<orkon> shs96c: you put the script to check to the global object
<orkon> q?
<orkon> ack next
<orkon> shs96c: so I think hit testing is sufficient for interactions, but you might get weird results for some elements
<orkon> shs96c: if you have a web component, how do you figure out what is interactable
<jgraham> q+
<orkon> ack next
<orkon> jgraham: yes, I agree with all of that
<orkon> jgraham: I think we should try to do nothing in terms of visibility and having smth in the API that directly exposes the pointer interactions APIs, and it makes sense to expose that
<orkon> q?
<orkon> q?
<whimboo> RRSAgent: make minutes
<RRSAgent> I have made the request to generate https://www.w3.org/2023/09/15-webdriver-minutes.html whimboo
jgraham commented 11 months ago

I think we should try to do nothing in terms of visibility and having smth in the API that directly exposes the pointer interactions APIs

i.e. instead of having visibility directly in the spec we should expose APIs like getBoundingClientRects and elementsFromPoint.