sync request failure

[szakeri]

OpenStack Release

mitaka

Description

This failure happened as part of running TLS api tests. Output of test:

_StringException: pythonlogging:'': {{{
2016-09-08 14:49:28,076 85605 INFO     [tempest.lib.common.rest_client] Request (TLSListenersTestJSON:test_create_listener_empty_uuid_tls_container): 404 POST http://10.190.4.152:9696/v2.0/lbaas/listeners 0.553s
2016-09-08 14:49:28,319 85605 INFO     [tempest.lib.common.rest_client] Request (TLSListenersTestJSON:test_create_listener_empty_uuid_tls_container): 200 GET http://10.190.4.152:9696/v2.0/lbaas/loadbalancers/db93604f-3cfe-4fd0-8a8c-6308b7bb155b/statuses 0.234s
2016-09-08 14:49:28,331 85605 INFO     [neutron_lbaas.tests.tempest.v2.api.test_listeners_tls:TLSListenersTestJSON] Finished: test_create_listener_empty_uuid_tls_container
}}}

Traceback (most recent call last):
  File "/Users/zakeri/GitHub/neutron-lbaas/neutron_lbaas/tests/tempest/v2/api/test_listeners_tls.py", line 203, in test_create_listener_empty_uuid_tls_container
    listener_ids=[self.listener_id])
  File "/Users/zakeri/GitHub/neutron-lbaas/neutron_lbaas/tests/tempest/v2/api/base.py", line 320, in _check_status_tree
    assert 'ONLINE' == load_balancer['operating_status']
_pytest.assertion.reinterpret.AssertionError: assert 'ONLINE' == 'DEGRADED'
  - ONLINE
  + DEGRADED

--- Error message in the f5-openstack-agent.log file:

2016-09-08 14:49:31.931 3235 DEBUG f5_openstack_agent.lbaasv2.drivers.bigip.plugin_rpc [-] f5_openstack_agent.lbaasv2.drivers.bigip.plugin_rpc.LBaaSv2PluginRPC method get_service_by_loadbalancer_id called with arguments (u'db93604f-3cfe-4fd0-8a8c-6308b7bb155b',) {} wrapper /usr/lib/python2.7/site-packages/oslo_log/helpers.py:45
2016-09-08 14:49:31.931 3235 DEBUG oslo_messaging._drivers.amqpdriver [-] CALL msg_id: f139ea438944459f8731f09404ee2100 exchange 'neutron' topic 'f5-lbaasv2-process-on-controller_Project' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:454
2016-09-08 14:49:31.980 3235 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: f139ea438944459f8731f09404ee2100 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:302
2016-09-08 14:49:31.981 3235 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.utils [-] sync request fc5dad9f-b37f-42cb-b66c-0b821d4619c4 FAILED
2016-09-08 14:49:31.982 3235 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.agent_manager [-] Exception: loadbalancer
2016-09-08 14:49:31.982 3235 DEBUG f5_openstack_agent.lbaasv2.drivers.bigip.agent_manager [-] currently known loadbalancer ids after sync: [u'db93604f-3cfe-4fd0-8a8c-6308b7bb155b'] sync_state /usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/agent_manager.py:504

Agent Version

9.0.2

[pjbreaux]

Adding this bug report in launchpad, as it might be applicable to our issue:

https://bugs.launchpad.net/neutron/+bug/1507050

Also a note from one of the neutron developers about operating_status:

DEGRADED basically means that one or more of its descendants
> are in an OFFLINE operating_status.  NO_MONITOR means no health monitor
> so operating_status can't be evaluated.  DISABLED means admin_state_up
> on that entity is set to False.

^--- from: http://lists.openstack.org/pipermail/openstack-dev/2015-December/082275.html

[mattgreene]

@szakeri: Could you please provide the failing test_id (name of test case) in order to recreate the problem and verify a fix?

[mattgreene]

Issue was due to envrionmental issues in the testbed. No longer seen.