Health Monitor still associated with LBaaS elements prevents stack removal

[ghost]

From: Darío Ezequiel Nievas
Date: Monday, September 21, 2015 at 4:17 PM
To: Douglas Wiegley
Subject: Heat + A10 LBaaS v1 driver - Error 33619969

Hi Doug, 
hope you are dowing 
We are using the A10 lbaas V1 driver with heat to provide autoscaling services.

Each time we try to delete a stack, we are getting te following Exception

2015-09-21 17:49:10.204 2429 ERROR neutron.api.v2.resource [req-7b5d1bb4-47c3-49e4-a210-36e3e38671bc ] delete failed
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource Traceback (most recent call last):
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/api/v2/resource.py", line 83, in resource
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     result = method(request=request, **args)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron/api/v2/base.py", line 490, in delete
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     obj_deleter(request.context, id, **kwargs)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron_lbaas/services/loadbalancer/plugin.py", line 242, in delete_pool
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     driver.delete_pool(context, p)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/lib/python2.7/dist-packages/neutron_lbaas/services/loadbalancer/drivers/a10networks/driver_v1.py", line 155, in delete_pool
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     self.a10.pool.delete(context, pool)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/a10_neutron_lbaas/v1/handler_pool.py", line 57, in delete
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     self.a10_driver.hm._delete(c, context, z)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/a10_neutron_lbaas/v1/handler_hm.py", line 72, in _delete
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     c.client.slb.hm.delete(self._meta_name(hm))
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/acos_client/v30/slb/hm.py", line 112, in delete
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     self._delete(self.url_prefix + name)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/acos_client/v30/base.py", line 41, in _delete
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     return self._request('DELETE', action, params, **kwargs)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/acos_client/v30/base.py", line 32, in _request
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     self.auth_header, **kwargs)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/acos_client/v30/axapi_http.py", line 123, in request
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     acos_responses.raise_axapi_ex(r, method, api_url)
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource   File "/usr/local/lib/python2.7/dist-packages/acos_client/v30/responses.py", line 147, in raise_axapi_ex
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource     raise ex(code, response['response']['err']['msg'])
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource InUse: 33619969 The health monitor cannot be deleted because it is still in use.
2015-09-21 17:49:10.204 2429 TRACE neutron.api.v2.resource 

These are the heat resources involved (showing just the relevant ones)

  pool:
    type: OS::Neutron::Pool
    properties:
      protocol: HTTP
      monitors: [{get_resource: monitor}]
      subnet: a10-subnet2
      lb_method: ROUND_ROBIN
      vip:
        protocol_port: 8080

  lb:
    type: OS::Neutron::LoadBalancer
    properties:
      protocol_port: 8080
      pool_id: {get_resource: pool}

  monitor:
   type: OS::Neutron::HealthMonitor
   properties:
     type: HTTP
     delay: 12
     max_retries: 5
     timeout: 10
     http_method: "GET"
     max_retries: 2
     timeout: 5
     url_path: "/"
     expected_codes: "200"

I'm also attaching the neutron-server log extract, for further details

Have you seen this problem before? I'm guessing that this is related with the way that heat handles dependencies when a stack delete is requested. 

Thanks in advance!

[ghost]

This was fixed in v2 but not v1.

[Cedev]

This is fixed in master (1.4 series, PR #144). There's a backport PR #154 to merge it into a 1.3 series driver.