search

F5Networks / f5-openstack-lbaasv2-driver

F5 LBaaSv2 service provider driver for OpenStack Liberty and beyond
http://clouddocs.f5.com/products/openstack/lbaasv2-driver/master/
Apache License 2.0
21 stars 37 forks source link

Assertion error in `test_policy_reject_multi_policy_multi_rules` #832

Closed ssorenso closed 6 years ago

ssorenso commented 6 years ago
Suggested Issue Type:

Test Bug, After talking with Paul, it came to my attention that there may be a timing issue with policies in general. This may impact ESD items as well.

Traceback
Traceback (most recent call last):
  File "/home-local/jenkins/workspace/openstack/driver/mitaka/12.1.2-undercloud-vlan/f5lbaasdriver/test/tempest/tests/api/test_l7policy_rules.py", line 306, in test_policy_reject_multi_policy_multi_rules
    assert not bigip_client.rule_exists("wrapper_policy",
_pytest.assertion.reinterpret.AssertionError: assert not True
 +  where True = <bound method BigIpClient.rule_exists of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f2802dab490>>('wrapper_policy', 'reject_1', ('Project_' + '5e331245540145e695990fb2465cac29'))
 +    where <bound method BigIpClient.rule_exists of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f2802dab490>> = <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f2802dab490>.rule_exists
 +    and   '5e331245540145e695990fb2465cac29' = <f5lbaasdriver.test.tempest.tests.api.test_l7policy_rules.L7PolicyJSONReject.test_policy_reject_multi_policy_multi_rules id=0x7f2802d878d0>.project_tenant_id
Impacted Tests

Expanded, direct impact:

Agent Version

mitaka

Operating System

Nightly

OpenStack Release

Mitaka

Bug Severity

Medium

Deployment

12.1.2 undercloud vlan

ssorenso commented 6 years ago

Other failure in this test that may be related:

  0 Traceback (most recent call last):
  1   File "/home-local/jenkins/workspace/openstack/driver/mitaka/11.5.4-undercloud-vxlan/f5lbaasd    river/test/tempest/tests/api/test_l7policy_rules.py", line 279, in test_policy_reject_multi_po    licy_multi_rules
  2     assert not bigip_client.rule_has_condition("wrapper_policy",
  3 _pytest.assertion.reinterpret.AssertionError: assert not True
  4  +  where True = <bound method BigIpClient.rule_has_condition of <f5lbaasdriver.test.tempest.s    ervices.clients.bigip_client.BigIpClient object at 0x7fd21e1b52d0>>('wrapper_policy', 'reject_    1', 'contains', 'es', ('Project_' + '89a6018f783d4877806ba8dd7eac22ba'))
  5  +    where <bound method BigIpClient.rule_has_condition of <f5lbaasdriver.test.tempest.servic    es.clients.bigip_client.BigIpClient object at 0x7fd21e1b52d0>> = <f5lbaasdriver.test.tempest.s    ervices.clients.bigip_client.BigIpClient object at 0x7fd21e1b52d0>.rule_has_condition
  6  +    and   '89a6018f783d4877806ba8dd7eac22ba' = <f5lbaasdriver.test.tempest.tests.api.test_l7    policy_rules.L7PolicyJSONReject.test_policy_reject_multi_policy_multi_rules id=0x7fd21e1c4450>    .project_tenant_id

They may be similar in that if a timing/race condition exists for this issue's original traceback, it could, logically, also exist for this traceback.

This is directly related to #839. So much so that #839 should be closed as a dup if this issue is mapped to #832

Test impact:

ssorenso commented 6 years ago

Yet another failure that may be timing related:

  0 Traceback (most recent call last):
  1   File "/home-local/jenkins/workspace/openstack/driver/newton/ha-11.6.1-overcloud/f5lbaasdriver/test/tempest/tests/api/test_l7policy_rules.py", line 415, in test_policy_reject_many_rules
  2     assert not bigip_client.rule_has_condition("wrapper_policy",
  3 AssertionError: assert not True
  4  +  where True = <bound method BigIpClient.rule_has_condition of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7fb03366ed90>>('wrapper_policy', 'reject_1', 'co    ntains', 'es', ('Project_' + '8816d28ac6634bc28c3b38791cea4efb'))
  5  +    where <bound method BigIpClient.rule_has_condition of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7fb03366ed90>> = <f5lbaasdriver.test.tempest.services    .clients.bigip_client.BigIpClient object at 0x7fb03366ed90>.rule_has_condition
  6  +    and   '8816d28ac6634bc28c3b38791cea4efb' = <f5lbaasdriver.test.tempest.tests.api.test_l7policy_rules.L7PolicyJSONReject.test_policy_reject_many_rules id=0x7fb033651090>.project_tenant_id
ssorenso commented 6 years ago

Another unique traceback:

  0 Traceback (most recent call last):
  1   File "/home-local/jenkins/workspace/openstack/driver/newton/12.1.2-undercloud-vlan/f5lbaasdr    iver/test/tempest/tests/api/test_l7policy_rules.py", line 266, in test_policy_reject_multi_pol    icy_multi_rules
  2     self._delete_l7rule(l7policy1.get('id'), self.rule1.get('id'))
  3   File "/usr/local/lib/python2.7/dist-packages/neutron_lbaas/tests/tempest/v2/api/base.py", li    ne 434, in _delete_l7rule
  4     cls._wait_for_load_balancer_status(cls.load_balancer.get('id'))
  5   File "/usr/local/lib/python2.7/dist-packages/neutron_lbaas/tests/tempest/v2/api/base.py", li    ne 308, in _wait_for_load_balancer_status
  6     operating_status=operating_status))
  7 tempest.lib.exceptions.TimeoutException: Request timed out
  8 Details: Wait for load balancer ran for 60 seconds and did not observe 56939d11-d24d-4370-be6b    -544a6361bde6 reach ACTIVE provisioning status and ONLINE operating status.
Associated test builds:
ssorenso commented 6 years ago

May also be an issue as this test failure seems to be intermittent:

  0 Traceback (most recent call last):
  1   File "/home-local/jenkins/workspace/openstack/driver/newton/ha-11.6.1-overcloud/f5lbaasdriver/test/tempest/tests/api/test_l7policy_rules.py", line 219, in test_policy_reject_three_rules
  2     assert not \
  3 AssertionError: assert not True
  4  +  where True = <bound method BigIpClient.rule_has_condition of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7fb9ef787b10>>('wrapper_policy', 'reject_1', 'co    ntains', 'es', ('Project_' + 'b0fccff9be474a6481ceed73f847f65e'))
  5  +    where <bound method BigIpClient.rule_has_condition of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7fb9ef787b10>> = <f5lbaasdriver.test.tempest.services    .clients.bigip_client.BigIpClient object at 0x7fb9ef787b10>.rule_has_condition
  6  +    and   'b0fccff9be474a6481ceed73f847f65e' = <f5lbaasdriver.test.tempest.tests.api.test_l7policy_rules.L7PolicyJSONReject.test_policy_reject_three_rules id=0x7fb9ef74cf50>.project_tenant_id
Test impact