F5Networks / f5-openstack-lbaasv2-driver

F5 LBaaSv2 service provider driver for OpenStack Liberty and beyond
http://clouddocs.f5.com/products/openstack/lbaasv2-driver/master/
Apache License 2.0
21 stars 37 forks source link

L7 tempest test fails asserting policy does NOT exist, after all rules are deleted from policy #415

Closed pjbreaux closed 7 years ago

pjbreaux commented 7 years ago

I believe this is a bug and not a test issue.

This is a reproducible issue against BIG-IP 11.5.4.

Agent Version

9.2.0

OpenStack Release

Mitaka

Description

The test api/test_l7policy_rules.py::L7PolicyJSONReject::test_policy_reject_many_rules creates many rules that vary the comparison type and the value, all wrapped in a reject policy. The REST calls and the tempest failure is given below. It appears that the policy still exists even after all rules are deleted, which should result in the empty policy being deleted (called policy_wrapper).

_StringException: pythonlogging:'': {{{
2017-02-02 16:09:27,471 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies.json 0.927s
2017-02-02 16:09:28,077 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.606s
2017-02-02 16:09:28,697 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.618s
2017-02-02 16:09:29,819 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 1.121s
2017-02-02 16:09:30,227 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.407s
2017-02-02 16:09:31,150 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 0.921s
2017-02-02 16:09:31,501 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.350s
2017-02-02 16:09:32,691 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 1.189s
2017-02-02 16:09:33,207 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.515s
2017-02-02 16:09:34,534 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 1.327s
2017-02-02 16:09:35,143 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.608s
2017-02-02 16:09:36,810 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 1.666s
2017-02-02 16:09:37,197 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.386s
2017-02-02 16:09:39,085 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 1.888s
2017-02-02 16:09:39,854 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.768s
2017-02-02 16:09:41,674 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 201 POST http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules.json 1.819s
2017-02-02 16:09:42,106 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.430s
2017-02-02 16:09:44,564 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/1c17735a-3078-40c9-9b20-596d00073fc6.json 1.918s
2017-02-02 16:09:45,091 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.526s
2017-02-02 16:09:46,715 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.621s
2017-02-02 16:09:48,559 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/7db33e85-09b2-4961-b75a-d08bab427a56.json 1.732s
2017-02-02 16:09:49,031 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.471s
2017-02-02 16:09:50,708 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.675s
2017-02-02 16:09:52,250 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/551990a3-e09e-4ba3-bad8-757f307509aa.json 1.540s
2017-02-02 16:09:53,166 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.915s
2017-02-02 16:09:54,702 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.533s
2017-02-02 16:09:56,341 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/375611b7-882b-457e-9ef7-6ab2977bea83.json 1.638s
2017-02-02 16:09:56,670 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.328s
2017-02-02 16:09:58,187 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.515s
2017-02-02 16:09:59,412 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/bc51a73f-7901-41d9-b417-06f1e53c1c5f.json 1.224s
2017-02-02 16:09:59,873 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.460s
2017-02-02 16:10:01,562 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.686s
2017-02-02 16:10:02,487 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/6d8d083c-ccdd-411d-8068-b585fd18eb68.json 0.923s
2017-02-02 16:10:02,784 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.296s
2017-02-02 16:10:04,160 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.374s
2017-02-02 16:10:05,355 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b/rules/e4c29778-ddf3-4cb0-a8ef-9a475be3a531.json 1.194s
2017-02-02 16:10:05,966 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.611s
2017-02-02 16:10:07,502 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.533s
2017-02-02 16:10:07,810 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:test_policy_reject_many_rules): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.307s
2017-02-02 16:10:08,732 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:tearDown): 204 DELETE http://10.190.4.140:9696/v2.0/lbaas/l7policies/2a59bba1-7c1d-43bf-871a-c6b2b5ef053b 0.858s
2017-02-02 16:10:09,039 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:tearDown): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.306s
2017-02-02 16:10:10,575 1771 INFO     [tempest.lib.common.rest_client] Request (L7PolicyJSONReject:tearDown): 200 GET http://10.190.4.140:9696/v2.0/lbaas/loadbalancers/73b3ad83-d3b3-4eed-8360-d9f55dce6deb 0.533s
2017-02-02 16:10:10,575 1771 INFO     [f5lbaasdriver.test.tempest.tests.api.test_l7policy_rules:L7PolicyJSONReject] Finished: test_policy_reject_many_rules
}}}

Traceback (most recent call last):
  File "/home/testlab/f5-openstack-lbaasv2-driver/f5lbaasdriver/test/tempest/tests/api/test_l7policy_rules.py", line 413, in test_policy_reject_many_rules
    assert not \
_pytest.assertion.reinterpret.AssertionError: assert not True
 +  where True = <bound method BigIpClient.policy_exists of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f9c247aca90>>('wrapper_policy', ('Project_' + 'c747d54ebb3a45ca8431da5bb40a3e3f'))
 +    where <bound method BigIpClient.policy_exists of <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f9c247aca90>> = <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f9c247aca90>.policy_exists
 +      where <f5lbaasdriver.test.tempest.services.clients.bigip_client.BigIpClient object at 0x7f9c247aca90> = <f5lbaasdriver.test.tempest.tests.api.test_l7policy_rules.L7PolicyJSONReject.test_policy_reject_many_rules id=0x7f9c247a87d0>.bigip_client
 +    and   'c747d54ebb3a45ca8431da5bb40a3e3f' = <f5lbaasdriver.test.tempest.tests.api.test_l7policy_rules.L7PolicyJSONReject.test_policy_reject_many_rules id=0x7f9c247a87d0>.project_tenant_id

Snapshot from the agent log:

Text: u'{"kind":"tm:ltm:virtual:policies:policiescollectionstate","selfLink":"https://localhost/mgmt/tm/ltm/virtual/~Project_c747d54ebb3a45ca8431da5bb40a3e3f~Project_26e71e02-051a-48da-8a6a-e685f46ff4d0/policies?ver=11.5.4","policiesReference":{"link":"https://localhost/mgmt/tm/ltm/virtual/~Project_c747d54ebb3a45ca8431da5bb40a3e3f~Project_26e71e02-051a-48da-8a6a-e685f46ff4d0/policies?ver=11.5.4","isSubcollection":true,"items":[{"kind":"tm:ltm:virtual:policies:policiesstate","name":"wrapper_policy","partition":"Project_c747d54ebb3a45ca8431da5bb40a3e3f","fullPath":"/Project_c747d54ebb3a45ca8431da5bb40a3e3f/wrapper_policy","generation":65,"selfLink":"https://localhost/mgmt/tm/ltm/virtual/~Project_c747d54ebb3a45ca8431da5bb40a3e3f~Project_26e71e02-051a-48da-8a6a-e685f46ff4d0/policies/~Project_c747d54ebb3a45ca8431da5bb40a3e3f~wrapper_policy?ver=11.5.4"}]}}' wrapper /usr/lib/python2.7/site-packages/icontrol/session.py:265
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver [req-78e7be32-c4f4-44f5-bd81-ccf7d1652d84 da0a8ac680dd42c4b0cad6554b2f9d7f c747d54ebb3a45ca8431da5bb40a3e3f - - -] 409 Unexpected Error: Conflict for uri: https://10.190.3.45:443/mgmt/tm/ltm/virtual/~Project_c747d54ebb3a45ca8431da5bb40a3e3f~Project_26e71e02-051a-48da-8a6a-e685f46ff4d0/policies/
Text: u'{"code":409,"message":"01020066:3: The requested virtual server policy (/Project_c747d54ebb3a45ca8431da5bb40a3e3f/Project_26e71e02-051a-48da-8a6a-e685f46ff4d0 /Project_c747d54ebb3a45ca8431da5bb40a3e3f/wrapper_policy) already exists in partition Project_c747d54ebb3a45ca8431da5bb40a3e3f.","errorStack":[]}'
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver Traceback (most recent call last):
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver   File "/usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/icontrol_driver.py", line 1216, in _common_service_handler
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver     all_subnet_hints)
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver   File "/usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/lbaas_builder.py", line 61, in assure_service
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver     self._assure_l7policies_created(service)
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver   File "/usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/lbaas_builder.py", line 415, in _assure_l7policies_created
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver     raise f5_ex.L7PolicyCreationException(err.message)
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver L7PolicyCreationException: 409 Unexpected Error: Conflict for uri: https://10.190.3.45:443/mgmt/tm/ltm/virtual/~Project_c747d54ebb3a45ca8431da5bb40a3e3f~Project_26e71e02-051a-48da-8a6a-e685f46ff4d0/policies/
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver Text: u'{"code":409,"message":"01020066:3: The requested virtual server policy (/Project_c747d54ebb3a45ca8431da5bb40a3e3f/Project_26e71e02-051a-48da-8a6a-e685f46ff4d0 /Project_c747d54ebb3a45ca8431da5bb40a3e3f/wrapper_policy) already exists in partition Project_c747d54ebb3a45ca8431da5bb40a3e3f.","errorStack":[]}'
2017-02-02 16:10:02.559 23186 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver
pjbreaux commented 7 years ago

It looks like the failure of this test leads into the failure of tests that are run after it. We must be more aggressive on the teardown of the tenant. Likely, this is due to a policy not being torn down. If we remove the policy, then we should be able to remove the tenant in most cases.

pjbreaux commented 7 years ago

@dflanigan: this was the issue which is causing further test failures and I was wrong--it's not a test bug. I'm looking at the issue currently to provide a more well-rounded reconstruction of the problem.

pjbreaux commented 7 years ago

For actual test issue, it appears that we are attempting to delete the policy before it has been disassociated from the virtual:

2017-02-07 22:19:26.758 55140 DEBUG root [req-f4036b27-b722-40fa-938a-90675ba5a332 833309c56e064131898d2e198425bd56 61345078b7414f9cad738455e6042c2e - - -] delete WITH uri: https://10.190.3.22:443/mgmt/tm/ltm/policy/~Project_61345078b7414f9cad738455e6042c2e~wrapper_policy/ AND suffix:  AND kwargs: {} wrapper /usr/lib/python2.7/site-packages/icontrol/session.py:257
2017-02-07 22:19:26.768 55140 DEBUG root [req-f4036b27-b722-40fa-938a-90675ba5a332 833309c56e064131898d2e198425bd56 61345078b7414f9cad738455e6042c2e - - -] RESPONSE::STATUS: 400 Content-Type: application/json Content-Encoding: None
Text: u'{"code":400,"message":"01070635:3: The policy (/Project_61345078b7414f9cad738455e6042c2e/wrapper_policy) is referenced by one or more virtuals.","errorStack":[]}' wrapper /usr/lib/python2.7/site-packages/icontrol/session.py:265
2017-02-07 22:19:26.769 55140 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver [req-f4036b27-b722-40fa-938a-90675ba5a332 833309c56e064131898d2e198425bd56 61345078b7414f9cad738455e6042c2e - - -] 400 Unexpected Error: Bad Request for uri: https://10.190.3.22:443/mgmt/tm/ltm/policy/~Project_61345078b7414f9cad738455e6042c2e~wrapper_policy/
Text: u'{"code":400,"message":"01070635:3: The policy (/Project_61345078b7414f9cad738455e6042c2e/wrapper_policy) is referenced by one or more virtuals.","errorStack":[]}'
pjbreaux commented 7 years ago

@jlongstaf: tested against 11.5.4 and did not see this issue. Could this be related to https://github.com/F5Networks/f5-openstack-agent/issues/577?

You said you tested against 11.6.0 and it failed?

pjbreaux commented 7 years ago

This error is two fold, I believe. It is associated with the issue linked above in the agent. It is primarily failing due to the error below, when deploying a condition that has similar operands as an existing condition:

2017-05-23 18:00:23.602 22387 DEBUG root [req-f52b770e-be8d-4c16-8cd9-9e8377aea71d a76cc34e2a3b40e29fcefb123f818b54 cfceb9cf401544359c553b855eba5a46 - - -] patch WITH uri: https://10.190.7.238:443/mgmt/tm/ltm/policy/~Project_cfceb9cf40154
4359c553b855eba5a46~wrapper_policy/ AND suffix:  AND kwargs: {'json': {'name': 'wrapper_policy', 'rules': [{'ordinal': 1, 'conditions': [{'name': '0', 'httpUri': True, 'request': True, 'endsWith': True, 'values': [u'real'], 'path': True},
 {'name': '1', 'contains': True, 'request': True, 'tmName': u'X-HEADER', 'values': [u'es'], 'httpHeader': True}, {'startsWith': True, 'name': '2', 'httpUri': True, 'request': True, 'values': [u're'], 'path': True}, {'startsWith': True, 'n
ame': '3', 'request': True, 'host': True, 'values': [u'real'], 'httpHost': True}, {'name': '4', 'httpUri': True, 'contains': True, 'request': True, 'values': [u'al'], 'path': True}, {'name': '5', 'httpUri': True, 'contains': True, 'reques
t': True, 'values': [u'l'], 'path': True}], 'name': u'reject_1', 'actions': [{'reset': True, 'forward': True, 'request': True, 'name': '0'}]}], 'partition': u'Project_cfceb9cf401544359c553b855eba5a46', 'controls': ['forwarding'], 'strateg
y': 'first-match', 'requires': ['http']}} wrapper /usr/lib/python2.7/site-packages/icontrol/session.py:257
2017-05-23 18:00:23.635 22387 DEBUG root [req-f52b770e-be8d-4c16-8cd9-9e8377aea71d a76cc34e2a3b40e29fcefb123f818b54 cfceb9cf401544359c553b855eba5a46 - - -] RESPONSE::STATUS: 400 Content-Type: application/json Content-Encoding: None
Text: u'{"code":400,"message":"010716d8:3: Compilation of ltm policies for virtual server /Project_cfceb9cf401544359c553b855eba5a46/Project_2df9f8a8-ec9e-4cb5-a2f6-2953a38e1b5c failed; Failed to compile the combined policies.","errorStack
":[]}' wrapper /usr/lib/python2.7/site-packages/icontrol/session.py:265
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver [req-f52b770e-be8d-4c16-8cd9-9e8377aea71d a76cc34e2a3b40e29fcefb123f818b54 cfceb9cf401544359c553b855eba5a46 - - -] 400 Unexpected Error: Bad Requ
est for uri: https://10.190.7.238:443/mgmt/tm/ltm/policy/~Project_cfceb9cf401544359c553b855eba5a46~wrapper_policy/
Text: u'{"code":400,"message":"010716d8:3: Compilation of ltm policies for virtual server /Project_cfceb9cf401544359c553b855eba5a46/Project_2df9f8a8-ec9e-4cb5-a2f6-2953a38e1b5c failed; Failed to compile the combined policies.","errorStack
":[]}'
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver Traceback (most recent call last):
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver   File "/usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/icontrol_driver.py", line 1250, in _common_service_handler
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver     all_subnet_hints)
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver   File "/usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/lbaas_builder.py", line 63, in assure_service
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver     self._assure_l7policies_created(service)
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver   File "/usr/lib/python2.7/site-packages/f5_openstack_agent/lbaasv2/drivers/bigip/lbaas_builder.py", line 462, in _assure_l7policies_created
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver     raise f5_ex.L7PolicyCreationException(err.message)
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver L7PolicyCreationException: 400 Unexpected Error: Bad Request for uri: https://10.190.7.238:443/mgmt/tm/ltm/policy/~Project_cfceb9cf401544359c553b
855eba5a46~wrapper_policy/
2017-05-23 18:00:23.636 22387 ERROR f5_openstack_agent.lbaasv2.drivers.bigip.icontrol_driver Text: u'{"code":400,"message":"010716d8:3: Compilation of ltm policies for virtual server /Project_cfceb9cf401544359c553b855eba5a46/Project_2df9f
8a8-ec9e-4cb5-a2f6-2953a38e1b5c failed; Failed to compile the combined policies.","errorStack":[]}'
pjbreaux commented 7 years ago

This is an issue in the BIG-IP 11.6.0 version we are testing against (BIGIP-11.6.0.0.0.401). A hotfix is available that addresses my second point above. The recommendation to users is to apply the hotfix, or bump up their BIG-IP version to 11.6.1.

Here's the release notes for the latest hotfix related that issue:

447043-3 : Cannot have 2 distinct 'contains' conditions on the same LTM policy operand

Component: Local Traffic Manager

Symptoms:
Cannot express conditions such as 'user-agent contains 'Android' AND 'Mobile'. LTM policies have operands that can be matched against a set of values, causing a match when the operand matches one of these values. There is no way to use current functionality to match all of the values. One specific situation in which this is needed is to configure 'contains'.

Conditions:
Specify an ltm rule with 2 conditions with the same operand and match type, for example:

           conditions {
                0 {
                    http-header
                    name User-Agent
                    contains
                    values { Android }
                }
                1 {
                    http-header
                    name User-Agent
                    contains
                    values { Mobile }
                }

Impact:
The policy does not work. The system posts an error message similar to the following: Failed to compile the combined policies.

Fix:
LTM policies now allow for rules to have multiple conditions on the same operand and same match type so that 'user-agent contains 'Android' AND 'Mobile' can now be expressed by specifying:

           conditions {
                0 {
                    http-header
                    name User-Agent
                    contains
                    values { Android }
                }
                1 {
                    http-header
                    name User-Agent
                    contains
                    values { Mobile }
                }