coinzdude
commented
2 years ago More commentary: In an ideal state, a type of 'protocol' for programmable interaction would exist from the WebDriver framework, to be followed by UI control developers to follow and implement, with the goal of facilitating UI recording and playback of the controls. This would require that some framework is established, and then followed by UI control developers. In the interim, JavaScript manipulation might provide a simple enough method to manage these controls for the foreseeable future.
toddtarsi
🚀 Feature Proposal
A framework to facilitate recording and playback of 'problematic' UI controls
Motivation
It is common in the modern web dev stack to leverage a number of controls and web UI libraries to improve the user experience. However, those controls are often not aligned with any particular concept of UI automation for test execution. As an example, Select2 is a highly popular UI control, but is not readily able to be recorded and used in script playback. Similarly, there are a number of other controls, date fields, tables, drag and drop libraries, and many more, as well as less obvious challenges like iframes and windows management, or often, custom developed UI controls.
To address these issues, Selenium script developers are often tasked with research and development to hopefully deploy some work-around script, some set of JS code which modifies the behavior of the underlying control to allow it to be used within the recording and playback process. As is often the case, this problem is made more challenging as browsers like Chrome have strict JS scope execution limitations, landing the dev with 'some working code' that can't be used from within the selenium framework.
Example
To continue with the Select2 control, there are at least two options. One, if you own the generating code, implement a flag to control select2 usage, enabled or disabled, and in recording and playback, fall back on a standard select list interface. Alternatively, if you can inject JavaScript into the web UI, you can attempt to disable select2 on the client side, and in the case of Firefox, you can use this JavaScript to facilitate test recording and playback. However, Chrome disallows this script scope execution and will not disable the controls in playback using, for example the Selenium IDE or Side Runner.
Additional notes
If Electron provides better opportunity at interacting with Chrome through WebDriver (And this might be the crux of the issue), can we then, implement a customizable "extension pack" for test recording and playback, which provides an opportunity to 'plug in' a behavior control (JavaScript) which overrides the browser behaviors to facilitate the test development. So in the general case, "Fix these select2s, date fields, new window openers, to be more record/playback friendly". If so, GitHub could provide a method to store and provide access to a library of developed scripts for common controls, Imagine "Select2 Nicer", to choose from when creating test scripts with the IDE.
Additionally, a selenium script developer could provide their own JS to manipulate what is needed easily for their own custom UI. It's a little clunky to do this through SIDE JS commands currently via the side script and editor, particularly as a non-developer, and might work more intuitively as an extension framework.