F5Networks / f5-openstack-lbaasv2-driver

F5 LBaaSv2 service provider driver for OpenStack Liberty and beyond
http://clouddocs.f5.com/products/openstack/lbaasv2-driver/master/
Apache License 2.0
21 stars 37 forks source link

Intermittent failure in functional test: test_balancer_updates_scans.py #83

Closed pskergan closed 8 years ago

pskergan commented 8 years ago

Agent and Driver Version

v8.0.1RC1

Operating System

CentOS 7

OpenStack Release

Liberty

Description

Openstack deployed as Overcloud with BigIP version 11.6.

This error occurred once -- on the first run of the test script -- but the next three trials showed no error.

test_balancer_updates_scans.py::TestPoolUpdateScenarios::test_pool_update_configs[id-4b620c56-65e8-4484-82d2-007d935dcd25_test] ERROR

=============================================================== ERRORS =============================================================== ____ ERROR at teardown of TestPoolUpdateScenarios.test_pool_update_configs[id-4b620c56-65e8-4484-82d2-007d935dcd25test] ____

def finalize():
    pp('Entered setup/finalize.')
    nclientmanager.delete_all_lbaas_healthmonitors()
  nclientmanager.delete_all_lbaas_pools()

../../../envs/overcloud/lib/python2.7/site-packages/f5_os_test/infrastructure.py:71:


../../../envs/overcloud/lib/python2.7/site-packages/f5_os_test/polling_clients.py:225: in delete_all_lbaas_pools self.delete_lbaas_pool(pool['id'])


self = <f5_os_test.polling_clients.NeutronClientPollingManager object at 0x10e451a50> pool_id = '04e04abf-a34b-4236-aecf-1571e370fc01'

def delete_lbaas_pool(self, pool_id):
    self.delete_all_lbaas_pool_members(pool_id)
    self._poll_call_with_exceptions(
        StateInvalidClient,
        super(NeutronClientPollingManager, self).delete_lbaas_pool,
        pool_id)
    attempts = 0
    while pool_id in\
            [p['id'] for p in super(NeutronClientPollingManager, self)
                .list_lbaas_pools()['pools']]:
        time.sleep(self.interval)
        attempts = attempts + 1
        if attempts > self.max_attempts:
          raise MaximumNumberOfAttemptsExceeded

E MaximumNumberOfAttemptsExceeded

../../../envs/overcloud/lib/python2.7/site-packages/f5_os_test/polling_clients.py:218: MaximumNumberOfAttemptsExceeded ------------------------------------------------------- Captured stdout setup -------------------------------------------------------- 'got here in the constructor' 'Entered setup/finalize.' -------------------------------------------------------- Captured stdout call -------------------------------------------------------- ‘update_lbaas_pool'

Deployment

Openstack deployed as Overcloud with BigIP version 11.6. tlc session: skergan_overcloud

Attachments

https://console.aws.amazon.com/s3/home?region=us-west-2#&bucket=f5-openstack-testing-logs&prefix=LBaaSv2/lbassv2-driver-issue-83/

zancas commented 8 years ago

I've seen something like this before. I suspect the root cause is a failure to remove vips from virtuals.

jlongstaf commented 8 years ago

This is a timing issue. Increasing the timeout interval in NeutronClientPollingManager allows the tests to pass. The interval is currently set to 0.4 s and needs to be longer. Setting to at least 1.5 s seems appropriate.

dflanigan commented 8 years ago

Now part of Tempest