search

SatelliteQE / robottelo

Robottelo is a test suite that exercises The Foreman.
GNU General Public License v3.0
61 stars 114 forks source link

Automation failure : test_positive_delete_bz1225588 #2483

Closed tkolhar closed 9 years ago

tkolhar commented 9 years ago

TestNames: tests.foreman.ui.test_org.Org.test_positive_delete_bz1225588

Error Message <selenium.webdriver.remote.webelement.WebElement object at 0x47f5110> is not None

Stacktrace

File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/abb3bf3f/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 375, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel66-ui/tests/foreman/ui/test_org.py", line 332, in test_positive_delete_bz1225588 self.assertIsNone(self.org.search(org_name)) File "/usr/lib64/python2.7/unittest/case.py", line 994, in assertIsNone self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib64/python2.7/unittest/case.py", line 450, in fail raise self.failureException(msg) '<selenium.webdriver.remote.webelement.WebElement object at 0x47f5110> is not None\n

tkolhar commented 9 years ago

this issue occurred again dated on 06/24/2015 TestNames: tests.foreman.ui.test_org.Org.test_positive_delete_bz1225588_1

Error Message <selenium.webdriver.remote.webelement.WebElement object at 0x5e2fcd0> is not None

Stacktrace

File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/ca586003/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 375, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel71-ui/tests/foreman/ui/test_org.py", line 332, in test_positive_delete_bz1225588 self.assertIsNone(self.org.search(org_name)) File "/usr/lib64/python2.7/unittest/case.py", line 994, in assertIsNone self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib64/python2.7/unittest/case.py", line 450, in fail raise self.failureException(msg) '<selenium.webdriver.remote.webelement.WebElement object at 0x5e2fcd0> is not None\n-------------------- >> begin captured logging << --------------------

abalakh commented 9 years ago

It appears that organization deletion is async operation, which takes some time. The test is failing as it's checking whether organization was deleted immediately after triggering the deletion. I've tried something simple like navigating to dashboard before checking organization, and it didn't help, deletion takes even longer. So the task here is to figure out the way to wait for complete organization deletion.

sthirugn commented 9 years ago

Failed in Snap 12

jyejare commented 9 years ago

Even I tried some alternate ways but no success. Added check for success notification and then look for the org is deleted or not. But deletion of org is very slow that our check is happening early, I will say very early. So just tried one trick. I am verifying the org is deleted or not for 5 times. I put that into loop of 5. And then I am asserting it as NONE.

for i in range(5):
      ack=self.org.search(org_name)
self.assertIsNone(ack)

And this is solving my problem 

[root@jyejare robottelo]# nosetests -m test_positive_delete_bz1225588 ./tests/foreman/ui/test_org.py
.
----------------------------------------------------------------------
Ran 1 test in 261.113s

OK
rplevka commented 9 years ago

I did some testing and found out, the issue occurs only when using nailgun to create the organization. One does not even need to upload a manifest nor create a LC environment to get the issue replicated. While I used robottelo.ui.factory.make_org, this issue happend 0 times:

        # note this is not an equivalent to the original test as I'm skipping uploading the manifest (just for the test to finish faster)
        with Session(self.browser) as session:
            make_org(session, org_name=org_name)
            make_lifecycle_environment(session, org_name, name='DEV')
            make_lifecycle_environment(
                session, org_name, name='QE', prior='DEV'
            )
            # Org cannot be deleted when selected,
            # So switching to Default Org and then deleting.
            session.nav.go_to_select_org('Default Organization')
            self.org.remove(org_name, really=True)
            self.assertIsNone(self.org.search(org_name))

$ nosetests -m bz12255 test_org.py
...
----------------------------------------------------------------------
Ran 3 tests in 117.289s

OK

UPDATE: it is a problem with 2 concurrent sessions generally, not just nailgun having the organization created (by using whatever method) in 1 session and then removing and filtering it using another one triggers the issue.

jyejare commented 9 years ago

@rplevka : Nice, then we are good to update this code. Can you please do that ?

tkolhar commented 9 years ago

this issue occurred again dated 08/03/2015 TestNames: tests.foreman.ui.test_org.Org.test_positive_delete_bz1225588_1

Error Message top_menu_locator not found: ('xpath', "//div[contains(@style,'static') or contains(@style,'fixed')]//li[contains(@class,'org-switcher')]/a")

Stacktrace File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/abb3bf3f/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 372, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel66-ui/tests/foreman/ui/test_org.py", line 332, in test_positive_delete_bz1225588 self.assertIsNone(self.org.search(org_name)) File "robottelo/ui/org.py", line 112, in search Navigator(self.browser).go_to_org() File "robottelo/ui/navigator.py", line 334, in go_to_org menu_locators['menu.any_context'], menu_locators['org.manage_org'], File "robottelo/ui/navigator.py", line 17, in menu_click u'top_menu_locator not found: {0}'.format(top_menu_locator)) 'top_menu_locator not found: (\'xpath\', "//div[contains(@style,\'static\') or contains(@style,\'fixed\')]//li[contains(@class,\'org-switcher\')]/a")\n--------------------

rplevka commented 9 years ago

@jyejare looking into the issue some more, it looks like my previous statement is False. The issue occurs every time user is trying to delete organization, no matter of how it was created. There is some time needed for indexing. Having some fixed delay in place seems to help, but the time needed for indexing varies.

So I guess your for loop should be fine (probably add a break statement inside for a case ack is already None). Anyway, I believe the behavior we experience should be filed as a bug.

tkolhar commented 9 years ago

this issue occurred again dated 08/05/2015 TestNames: tests.foreman.ui.test_org.Org.test_positive_delete_bz1225588_1

Error Message

<selenium.webdriver.remote.webelement.WebElement object at 0x57b2cd0> is not None

Stacktrace

File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/1b673361/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 372, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel67-ui/tests/foreman/ui/test_org.py", line 252, in test_positive_delete_bz1225588 self.assertIsNone(self.org.search(org_name)) File "/usr/lib64/python2.7/unittest/case.py", line 994, in assertIsNone self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib64/python2.7/unittest/case.py", line 450, in fail raise self.failureException(msg) '<selenium.webdriver.remote.webelement.WebElement object at 0x57b2cd0> is not None\n--------------------

tkolhar commented 9 years ago

this issue failed again dated 08/07/2015 TestNames: tests.foreman.ui.test_org.Org.test_positive_delete_bz1225588

Error Message <selenium.webdriver.remote.webelement.WebElement object at 0x4978c10> is not None

Stacktrace

File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/home/jenkins/shiningpanda/jobs/1b673361/virtualenvs/d41d8cd9/lib/python2.7/site-packages/ddt.py", line 146, in wrapper return func(self, _args, _kwargs) File "robottelo/common/decorators.py", line 372, in wrapper_func return func(_args, _kwargs) File "/home/jenkins/workspace/satellite6-automation-downstream-rhel67-ui/tests/foreman/ui/test_org.py", line 254, in test_positive_delete_bz1225588 self.assertIsNone(self.org.search(org_name)) File "/usr/lib64/python2.7/unittest/case.py", line 994, in assertIsNone self.fail(self._formatMessage(msg, standardMsg)) File "/usr/lib64/python2.7/unittest/case.py", line 450, in fail raise self.failureException(msg) '<selenium.webdriver.remote.webelement.WebElement object at 0x4978c10> is not None\n--------------------

jyejare commented 9 years ago

@rplevka : Updated the code as I suggested. Please review.