Add support for robotframework-seleniumtestability

[datakurre]

https://pypi.org/project/robotframework-seleniumtestability/ should add implicit wait for various browser actions to make testing more robust and faster (less sleep keywords required).

This implementation would make testability optional and active with

*** Variables ****************************************************************

${SELENIUM2LIBRARY_PLUGINS}  SeleniumTestability;True;7 Seconds;True

BUT. I HAVE NOT YET CONFIRMED TESTABILITY TO REALLY WORK. Instead I see related JavaScript errors in geckodriver.log. /cc @fredvd

Requires additional versions for buildout.coredev

robotframework-seleniumtestability = 2.0.0
furl = 2.1.3
orderedmultidict = 1.0.1

[mister-roboto]

@datakurre thanks for creating this Pull Request and helping to improve Plone!

TL;DR: Finish pushing changes, pass all other checks, then paste a comment:

@jenkins-plone-org please run jobs

To ensure that these changes do not break other parts of Plone, the Plone test suite matrix needs to pass, but it takes 30-60 min. Other CI checks are usually much faster and the Plone Jenkins resources are limited, so when done pushing changes and all other checks pass either start all Jenkins PR jobs yourself, or simply add the comment above in this PR to start all the jobs automatically.

Happy hacking!

[fredvd]

Testing it now on a PR with plone.schemaeditor where a modal might not be closed properly because of still running requests. But lets first see if the testability plugin doesn't break other robottests. First attempt:

https://jenkins.plone.org/job/pull-request-6.0-3.8/1469/

[fredvd]

@datakurre @petschki Build succeeded. But no clue if it actually 'did' something or helped on schema-editor robot tests only. We could re-run with the plugin enabled on all robot tests and see what happens.

[fredvd]

Started: https://jenkins.plone.org/job/pull-request-6.0-3.8/1471/

[fredvd]

1471 gave one test failure, about not beeing able to start headless chrome or the webdriver .

Setup failed:
WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (chrome not reachable)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:

https://jenkins.plone.org/job/pull-request-6.0-3.8/1471/testReport/junit/Products.CMFPlone.tests.test_robot/RobotTestCase/Scenario_Disable_Standard_Editor_in_the_Editing_Control_Panel/

I've started 3 same jobs now that run on each of the 3 worker nodes. Let's see if this occurs more often.

[mauritsvanrees]

I must say, if this library keeps reporting a warning like this for every robot test, I would rather not use it:

[ WARN ] Message: javascript error: Cannot read properties of undefined (reading 'when')
JavaScript stack:
TypeError: Cannot read properties of undefined (reading 'when')
    at eval (eval at executeAsyncScript (:552:26), <anonymous>:5:28)
    at eval (eval at executeAsyncScript (:552:26), <anonymous>:8:8)
    at executeAsyncScript (<anonymous>:552:47)
    at <anonymous>:567:29
    at callFunction (<anonymous>:446:22)
    at <anonymous>:460:23
    at <anonymous>:461:3
  (Session info: headless chrome=102.0.5005.61)
Stacktrace:
0   chromedriver                        0x0000000101cd8d19 chromedriver + 5197081
1   chromedriver                        0x0000000101c66b23 chromedriver + 4729635
2   chromedriver                        0x00000001018442b8 chromedriver + 393912
3   chromedriver                        0x000000010184789b chromedriver + 407707
4   chromedriver                        0x0000000101847c5b chromedriver + 408667
5   chromedriver                        0x00000001018aa27b chromedriver + 811643
6   chromedriver                        0x0000000101896802 chromedriver + 731138
7   chromedriver                        0x00000001018a92a1 chromedriver + 807585
8   chromedriver                        0x00000001018966f3 chromedriver + 730867
9   chromedriver                        0x000000010186ca49 chromedriver + 559689
10  chromedriver                        0x000000010186da75 chromedriver + 563829
11  chromedriver                        0x0000000101caabdd chromedriver + 5008349
12  chromedriver                        0x0000000101cafb64 chromedriver + 5028708
13  chromedriver                        0x0000000101cb4bcf chromedriver + 5049295
14  chromedriver                        0x0000000101cb07ca chromedriver + 5031882
15  chromedriver                        0x0000000101c8a59f chromedriver + 4875679
16  chromedriver                        0x0000000101cca388 chromedriver + 5137288
17  chromedriver                        0x0000000101cca50f chromedriver + 5137679
18  chromedriver                        0x0000000101ce0085 chromedriver + 5226629
19  libsystem_pthread.dylib             0x00007fff6d20c109 _pthread_start + 148
20  libsystem_pthread.dylib             0x00007fff6d207b8b thread_start + 15

It might be this line:

window.testability.when.ready

That would mean that testability is not actually available. But could be something else.

[datakurre]

Indeed. I can look into those issues with our setup as soon as I can (I am a sure I have enough time next week), and even then would make this opt-in only. For me it seemed that it did not really work and warnings may be result of that. I have good experience of testability elsewhere and was surprised of these issues.