Support for Native Browser Events

brian-mann commented 7 years ago

click

[x] simulated click fixes #2956
[x] on text-editable click in center move the cursor to the end
- [x] ensure possible in IE11 without causing side effects
[x] cy.body().type():
- blur focused element, send events to body
[x] support .rightClick() :new:

possible:

[ ] move cursor to beginning if click('left') or click('topleft') :warning:
not fixed:
[ ] document.execCommand("copy") does not work in cypress #2851 (execCommand will not work with untrusted events)

type

[x] by default issues native events
[x] force:true skips actionability and uses simulated keys
[ ] simulated keys have no delay :warning: change from 50ms
[x] native keys by default have no delay :warning: change from 50ms
[x] support new special .type() sequences: {moveToEnd},{moveToStart} :new:
[x] support new special .type() sequences: {tab},{shift+tab} :new:
[x] type follows focus :warning:
[x] trusted native keyboard events :new:
- checkValidity() returns true instead of false for min length on input #1930
  {moveToEnd}/{moveToStart}
IE: setSelectionRange on all inputs
chrome/ff: setSelectionRange except for email/number inputs. For those we use execCommand('selectAll') + selection.modify('...')
branch logic based on browserFeatures object. For ex)
```
{
setSelectionRangeOnAllInputTypes: true
}
```

focus

.focus behavior on certain text editable elements
- [ ] command should first focusable element :warning: change from subject
- [ ] focus host contenteditable #2717
  contenteditable
treated as any other text-editable in the commands above.
By default move cursor to end
[x] support {moveToEnd}/{moveToStart}
[x] support {tab}

file uploads :new:

proposal https://github.com/cypress-io/cypress/issues/170
[ ] tbd

cy.hover :new:

proposal https://github.com/cypress-io/cypress/issues/10
[ ] tbd

scroll behavior

[ ] scroll into view only if needed :warning:
- use getElementFromPoint on the pixel needing click
[ ] scroll into center by default :warning:
[ ] allow user to configure scroll position (center, bottom, top) Enable users to change the scrolling strategy #871 :new: For IE:
- scrollIntoView can't center, so calculate coords and try to center it manually

Considerations / Research

select text command? :new:

allow user to make arbitrary text selection

mouse state

[ ] tbd

~for mouse actions that involve mouse state:~

in open mode, warn if potentially affected test
includes click (since hover before click), hover, drag-and-drop

For IE11 & firefox, see branch issue-311-ie-ff

brian-mann commented 7 years ago

With the release of Chrome 63 we can now finally do this.

We'll begin by adding { native: true } to existing commands for users to opt into using the native variants of them.

From there we'll add new commands for file uploads, etc.

We can also expose a low level command for talking directly to the debugger protocol to expose all of the other power and goodness that extends well beyond WebDriver.

bahmutov commented 7 years ago

👏👍🎉

Sent from my iPhone

On Oct 26, 2017, at 18:56, Brian Mann notifications@github.com wrote:

With the release of Chrome 63 we can now finally do this.

We'll begin by adding { native: true } to existing commands for users to opt into using the native variants of them.

From there we'll add new commands for file uploads, etc.

We can also expose a low level command for talking directly to the debugger protocol to expose all of the other power and goodness that extends well beyond WebDriver.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

mateatslc commented 7 years ago

good news, sounds exciting! 👏

dziamid commented 7 years ago

Awesome! Is it devtools compatible? Do you plan to eventually deprecate the not-native commands?

brian-mann commented 7 years ago

It is dev tools compatible.

We will definitely not ever deprecate the non-native commands because they are actually more precise (as long as they work correctly). Non native can be issued synchronously and it's impossible to miss coordinates.

Native events will always be async, so if we ask for them to fire at a specific coordinate, it will take take before it's actually issued. If there are state changes by then - you have a failure.

We'll likely have a few native commands only like .attach() or something. But then enable you to opt into the native commands for commands we already have.

dwelle commented 7 years ago

Does this mean we'll able to invoke and test :hover/:active pseudo classes?

brian-mann commented 7 years ago

Yes

zth commented 6 years ago

Is there a plan to integrate file uploading better with native events landing? There's a comment about it here cut I cannot seem to find any more information about it.

atomboulian commented 6 years ago

My question is the same as @zth. Seems like this would be true, but I just want to confirm. There are medium articles talking about file uploads not being supported, even in February of this year.

jpike88 commented 6 years ago

Just curious where things are at? I can't get a good reading on how close this functionality is to being ready or if it is already available

brian-mann commented 6 years ago

@Bkucera is starting on this work this week. We've gone through the game plan and will begin implementing this and all of the other work orthogonal to native events.

egucciar commented 6 years ago

@brian-mann will this change some of the known "permanent limitations" of Cypress? One example being something like performance testing, which with integration with Dev Tools could totally be within reach

brian-mann commented 6 years ago

@egucciar yes, that page is a bit outdated now. I wouldn't say it's really native events per-se, we've known about them and tapped into the early parts of them for awhile, it has to do with the underlying architectural model and the proxy layer that cypress sits in.

There are a handful of things that all together will improve that and mean that Cypress will be a bit more transparent so your application behaves more closely to how it operates outside of Cypress. It's a bit like the uncertainty principle that by inspecting and viewing "the truth" of what happens in the browser in order to model and predict it, we thereby end up changing its behavior. Performance testing bit further outside-in than how Cypress operates, and because of that, our presence changes the behavior.

It is certainly possible for us to change even that - because the tools already exist, but it's more of a documentation / process problem for us. We're really good at one thing, and attempting to do too many things (even those closely related) is difficult while also maintaining a clear and specific objective.

egucciar commented 6 years ago

@brian-mann appreciate the detailed explanation! So in short though this may open up the possibility of further integrations, some use cases still may not be applicable due to the overall architecture, performance testing being one of those things. Makes perfect sense.

brian-mann commented 6 years ago

Updates: along with native events, this goes hand in hand with cross browser support, which I added an update for here: https://github.com/cypress-io/cypress/issues/310#issue-191595673

TLDR; we are adding cross browser support as a major priority here. Work has been going on for months and we're ready to begin implementation.

pvdyck commented 6 years ago

@bahmutov ready for Sprint 5 or postpone ?

brian-mann commented 5 years ago

Here are my thoughts from approximately Nov 2018

actionability algorithm changes

Brian Thoughts:

Whenever we fire an event manually (or possibly whenever we dispatch a native event) that would otherwise cause the browser to receive an event and allow for user code to respond to it...
- we always ask the "current state" of the browser rather than hold direct references to active elements or references...
- this is good, however, modern JS frameworks may have "reacted" to the synchronous event, but not yet dispatched their changes, which are typically enqueued via requestAnimationFrame
- this can lead to a problem where we "assume" the state of the DOM hasn't changed, and then begin dispatching events, but then asynchronously the JS application responds and does something unexpected
- technically, there's nothing wrong with this- if a user were to dispatch their action so fast that it happened prior to the JS reconciliation: "whatever happens, happens" is true
- but its highly unlikely because when we're automating, it's happening synchronously in a way that a human cannot replicate
- in order to solve this (and other things) we can actually tap factor this into our algorithm in a few ways...
  1. if we have an expected "whitelisted" set of events that we know will fire, we could then know if listeners were triggered in response to those. If listeners were triggered AND they called anything asynchronous (setTimeout, setInterval, requestAnimationFrame, etc) we could automatically await those timers before continuing on. This is ideal, but its a scalpel and it may be a whack-a-mole of edge cases, which brings us to #2
  2. instead of whitelisting specific "events", we simply attach to all events synchronously, and the moment that any event fires, we begin listening for any async calls. if after the event bubbling phase is complete and there were any async calls in response to it, then we await them even if they aren't specific to our event. because listeners have to be attached synchronously, we can likely get away with this algorithm without being coupled to specific actions or listeners. to do this requires having "book ends" on either side of the dispatched action. we need be the first one who receives the initial event (before other event listeners) and we need to be the very last one called in the bubble or capture phase so we know if async methods were called.
  3. there may actually be an even easier way.... instead of needing to insert bookends for the event phases, by overwriting the async functions on the browser, we may immediately receive a stace trace at the callsite and be able to programmatically figure out if its in response to an event handler. this would be easiest and would prevent us needing to know the exact moment an async event was dispatched or received by the browser.

Summary: To summarize the above: what we're doing is adding a reconciliation phase that awaits enqueued asynchronous functions (below a certain threshold) to fire in response to mutative events that we fire as part of our dispatching and actionability algorithms. This will ensure that JS app code "receives" the events and that the DOM is updated in response to those (if necessary). We are not awaiting subsequent async functions like network requests that happen as a side effect. Only DOM will be truly awaited in response to event handler callbacks (whether those JS app handlers are invoked synchronously or asynchronously).

Update: If nested async handlers are called (such as a recursive setTimeout) or requestAnimationFrame - since those are wrapped and known to be in response to the awaited original async handlers, we simply continue to aggregate time until a certain threshold is met: 100ms or 200ms or something configurable. At that time (or at such time that we discern that the DOM has flushed appropriately) we will no longer await nested functions. To be clear though, only timers and animation frames will be awaited, not things like network request.

Another idea, possibly even easier would be to listen to DOM mutation events, and to always flush at that time irrespective of the nested async calls if we find that it's a more reliable way to know whether or not we can move on.

tnrich commented 5 years ago

With the release of Chrome 63 we can now finally do this.

We'll begin by adding { native: true } to existing commands for users to opt into using the native variants of them.

From there we'll add new commands for file uploads, etc.

We can also expose a low level command for talking directly to the debugger protocol to expose all of the other power and goodness that extends well beyond WebDriver.

Are we able to use this already with the latest cypress @brian-mann ?

Any word on when we can trigger copy/paste? https://github.com/cypress-io/cypress/issues/2386 ?

Thanks for the hard work!! I'm excited for this to come to fruition!

ArCiGo commented 5 years ago

I was taking some notes about the missing things in the tool, but I'm not sure what of the following are in progress in the roadmap:

Handle of tabs, iframes and windows (open, close, switching between windows/iframes/tabs, etcetera)

In some cases it's necessary to switch between those elements in order to validate if the new page opened load successfully (or to interact with the HTML elements). Some months ago, a lot of sites had the old way to authenticate a user using the Google API for login, and this opened a new window to set the login credentials of the user.
Upload and download of files
Drag and drop
Page and elements with animations

By the way, nice tool!

WiseBird commented 5 years ago

@ArCiGo check this for the first question

ArCiGo commented 5 years ago

@WiseBird I updated my comment

devtreehouse commented 5 years ago

For our current project, this is a major issue for our test coverages compared to using selenium. While I totally see other issue in the road map as important as well, do we have a timeline when this feature will be available?

avallete commented 4 years ago

For our current project, this is a major issue for our test coverages compared to using selenium. While I totally see other issue in the road map as important as well, do we have a timeline when this feature will be available?

I totally agree, I really do love Cypress solution very much. But as it is now, and for what I've tested it, I didn't be able to really use it to implement e2e testing on an existing product without facing continuous bugs due to the ways events are handled. I've identified two bottleneck when trying to implements tests who came to me over and over:

Non-native events is one of the major ones, making emulating real user interaction very painful (hovering an element with your mouse, sending tab keys to the page, ...)
The other issue is pretty related to the first one, but basically is due to the way event are emulated within Cypress and the typing of those issue here. This kind of error is less present, but I saw it almost every time I was working on a project written in TypeScript.

As stated in numerous issues related to this one, e2e testing should be as close as possible to the final user behavior. Cypress have a really great interface and debugging environment which is really good sugar when your writing tests. But the way is behave is, for now, too far from the 'real user way' that WebDrivers (or even library like puppeteer) simulate. Making it really difficult to achieve 80% e2e tests coverage on an existing product with it.

So, I will also be really interested to know when this feature is planned, and if anything can be done by the community to contribute to it.

kilrain commented 4 years ago

Piling onto this mega-ticket: It'd be really great if Cypress fired the beforeInput event for cy.type. Without beforeinput support, my team is blocked from adopting Cypress as we use the Slate.js contenteditable library.

Related Slate ticket

From the docs;

beforeinput is not fired even though it is in the spec because no browser has adopted it.

The above isn't true... Chrome, Edge and Safari have all implemented the spec and Firefox is well on their way.

reydelo commented 4 years ago

@kilrain Hopefully the Cypress team will update their input commands to dispatch the beforeinput event, but until they do I’ve created a couple of simple custom commands which will trigger Slate’s input event listeners and make it respond https://github.com/ianstormtaylor/slate/issues/3476#issuecomment-617594068

devtreehouse commented 4 years ago

I saw this item being taken down from cypress road map page. Does that mean this item is not going to be brought up anytime soon?

jennifer-shehane commented 4 years ago

@devtreehouse Having native browser events support is not in current development. It is still part of our future roadmap.

We updated our published roadmap to reflect what is in active development - not necessarily every feature we plan to do in the future, but we intend to improve upon this communication in the coming months.

dwelle commented 4 years ago

@jennifer-shehane if Cypress exposed Chrome Devtools Protocol API (as suggested in https://github.com/cypress-io/cypress/issues/5670#issuecomment-552980964), then we could implement something in userland.

dmtrKovalenko commented 3 years ago

Published a package that implements native events using CDP connection (via cypress.automation hook). This should be a great start for native events at cypress 🐣.

https://github.com/dmtrKovalenko/cypress-real-events

moritzWa commented 3 years ago

For those that found themselves here looking for a solution: https://github.com/Bkucera/cypress-plugin-tab

dmtrKovalenko commented 3 years ago

@moritzWa it is not a real native browser event, so it won't work as a real browser tab. In order to make a real tab event use https://github.com/dmtrKovalenko/cypress-real-events and cy.realPress("Tab")

rsudarson commented 3 years ago

@dmtrKovalenko - Do cypress-real-events has support for drag and drop?

Badlapje commented 3 years ago

Would also really love this feature. Using cypress to test accessibility is not practical without native events.

Also: i cannot believe cypress not using native events isn't mentioned in the docs. There are several places where this should be made more clear. Eg. the page about cy.type. That page gives the impression you can emulate user behaviour, but for a11y purposes that's not true. I cannot select a label / button / link and type enter ... then get consistent results. See this ticket for more info.

beaugunderson commented 3 years ago

@Badlapje your second link is the same as the first :)

Badlapje commented 3 years ago

fixed it, thx @beaugunderson !

yanfengliu commented 3 years ago

Any progress on this? The minLength validation for <input> tag still cannot be triggered in Cypress 6.1

dmtrKovalenko commented 3 years ago

Mostly all native events are implemented at https://github.com/dmtrKovalenko/cypress-real-events

laerteneto commented 3 years ago

Hey guys, do we have some updates on this? I am curious to see when we will get this natively implemented. https://github.com/dmtrKovalenko/cypress-real-events looks very interesting indeed.

jennifer-shehane commented 3 years ago

Check our roadmap for what we're currently working on. We're working on some larger features now, but this isn't one of them unfortunately.

We suggest using https://github.com/dmtrKovalenko/cypress-real-events in the meantime.

ronit15 commented 3 years ago

In what version cy.type('{tab}') is supported? @eventualbuddha

hg13190 commented 2 years ago

For interacting with native elements while using cypress I have used a work around via which I was able to achieve it, by running tests in headed mode (or via cypress runner) and robotjs. Email me on hag13190@gmail.com to discuss details

mryellow commented 2 years ago

An interim release which delivers tab support would probably be a good idea. Leaving such a critical feature which has been demanded for 6 years hanging while other things are added is non-ideal.

gyurielf commented 1 year ago

An interim release which delivers tab support would probably be a good idea. Leaving such a critical feature which has been demanded for 6 years hanging while other things are added is non-ideal.

And it's still actual. :(

rkrisztian commented 1 year ago

I recently heard about WebDriver BiDi aiming to become the future of cross-browser automation, and it's something Cypress might want to consider for future implementation of native events. As we know, CDP was not really made for testing, and we're lucky that it's also supported in Firefox. But if for example, we want to support other browsers like Safari, then CDP won't be enough. (Plus, I also think that native events reflect more reliably what a real user does, and I find more value in realistic testing, but with the super handy UI of Cypress.)

I know that Cypress does not want to be Selenium, however they do not want to compete with other tools but rather collaborate with them. For more info please see: https://youtu.be/lpWwcWsyjQs?t=1310

MikeMcC399 commented 2 weeks ago

See also https://github.com/cypress-io/cypress/issues/30219

cypress-io / cypress