Airgun tests are slow when used on remote webdriver

[swadeley]

The web UI tests are too slow and some are currently not achieving anything, other than slowing down the unit under test, because they fail due to time sensitive tests taking longer to set up then the time allowed. For example, sync plan tests.

The airgun tests send one character at a time when filling in text fields. The code should be improved to send one string a time.

I have been told this is likely an upstream issue, in which case please file bug or raise issue there and link from here for tracking.

Thank you

[oshtaier]

1) Sync plan tests slowness has nothing to do with airgun framework. It is naturally that you need to wait until some events happen in future. If set up fail then logic just implemented in a bad way and need to be fixed on robottelo side 2) Airgun is not sending characters, but pure selenium does it as you can see it there: https://github.com/RedHatQE/widgetastic.core/blob/master/src/widgetastic/widget/input.py#L64 There is a chance that it can be improved or already improved in newer version of selenium (as we linked to static version according to https://github.com/SatelliteQE/robottelo/blob/master/requirements.txt#L15), but such change always require full re-test. Anyway impact of that problem is absolute minimum

[oshtaier]

Overall, it is expected that WebUI is always work slower than any other non-UI interaction like REST API or Hammer CLI. Also, we put the most complex system tests only in webUI part which take most execution time. One RHEL repo sync operation can take more time than whole module for other tests, so sometimes performance degrade on other things than clicks or fills

[oshtaier]

We definitely have things to improve like navigation or read of specific entries, not whole pages, but we need proper priority to work on this items

[oshtaier]

so @swadeley please provide exact issues with airgun framework (not robotello, not navmazing, not widgetastic, not selenium) that you like to be fixed

[rplevka]

re: text input - I think that in old-fashioned way we used to use (robottelo.ui), the text is pasted to the fields instantly, so I believe the "issue" (it's probably treated as a feature) gets implemented on a widgetastic or navmazing layer.

simple robottelo tests demonstrate it well:

from airgun.session import Session
from robottelo.ui.session import Session as Session2
from robottelo.test import UITestCase

def test_login_airgun():
    with(Session('login', 'admin', 'changeme')):
        pass

class LoginTestCase(UITestCase):
    def test_login(self):
        with(Session2(self)):
            pass

Note that in order to make the 'old-fashioned' test to work, one needs to comment out a code section from robottelo/ui/session.py:179 (we apparently stopped keeping the code compatible).

Running these tests in parallel reveals that airgun is slower in populating the form and sending it. The reason the old-fashion-style test is slower is because it uses different logic for determining the page finished loading (it makes it to the dashboard faster).

[swadeley]

so @swadeley please provide exact issues with airgun framework (not robotello, not navmazing, not widgetastic, not selenium) that you like to be fixed

@oshtaier I will leave this issue for others to discuss the work that needs to be done to make the test system faster. I just gave example of Sync Plans failing because system is too slow, not that they themselves are sending too much text.

Thank you

[JacobCallahan]

I don't know if the issue is with Airgun or one of its dependencies, but I have also noticed the extreme slowness of the tests. I'm sure there are a few things going on here, but there are definitely areas for improvement.

For example, the most recent run of tests.foreman.ui_airgun.test_activationkey.test_positive_create_with_envs took over 9 minutes to finish and executed 1725 events! This is a test that should just be creating an activation key, specifying a LE and CV, then verifying it was created with those.

Most of the noise seems to come from the polling that happens every second.