Load balancer enters into 'PENDING_UPDATE' state infinitely after a sequence of listener and loadbalancer updates

[ssorenso]

• Title: Load balancer enters into 'PENDING_UPDATE' state infinitely after a sequence of listener and loadbalancer updates

• Attachments: bug1.txt

• Details: This is a bug

Agent Version

f5-icontrol-rest (1.1.0) f5-openstack-agent (9.2.0b1) f5-openstack-lbaasv2-driver (9.2.0b1) f5-sdk (2.1.0)

Operating System

3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

OpenStack Release

mitaka

Description

The issue here is that the loadbalancer entered into 'PENDING_UPDATE' state after several update commands to the loadbalancer after a listener was configured. This locked out any additional update commands or delete commands to the listener, balancer, or subnet after the fact.

Deployment

See Agent Version section

[richbrowne]

Steve, thanks for the write up on this bug. Would you be able to provide a log file on the agent for us?

[ssorenso]

Sure. The attached is the truncated logs for recreating this bug. truncated_log.txt

My guess, if I were to make a 'thousand mile high' attempt, I would say that it's likely due to the fact that my subnet is: $ neutron subnet-show my_sub +-------------------+------------------------------------------------+ | Field | Value | +-------------------+------------------------------------------------+ | allocation_pools | {"start": "10.22.22.22", "end": "10.22.22.22"} | | cidr | 10.22.20.0/22 | | created_at | 2017-01-11T23:19:22 | | description | | | dns_nameservers | | | enable_dhcp | False | | gateway_ip | 10.22.22.0 | | host_routes | | | id | c074a886-d84f-4dd3-8b48-11b9bc35cd75 | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | my_sub | | network_id | 34d14da2-056d-4211-8532-c6fc62c3160a | | subnetpool_id | | | tenant_id | d1b2cff21a3344b787f31f2c8c4219a5 | | updated_at | 2017-01-11T23:19:22 | +-------------------+------------------------------------------------+

This subnet is only 1 address in size.

[ssorenso]

Once the rename has been performed, the loadbalancer config is locked and has to be cleaned up manually. Once the loadbalancer is created, the loadbalancer remains in an ERROR state (thus the suspicion of the subnet). We were unable to recreate this with a large subnet.

[ssorenso]

I was able to re-create this issue without the rename of the loadbalancer. One can simply stop at the creation of the listener: re_create.zip

The bug results in a degraded state where the stack is not able to create any new loadbalancer even after deleting the degraded loadbalancer: symptom.zip

The symptoms or resulting state makes it so that you cannot create a new loadbalancer without it being immediately placed into a provisioning state of 'ERROR', which results in a 'PENDING_UPDATE' state with any update whether it's a rename or otherwise.

[ssorenso]

I discovered the there was a tenantry mismatch in the bug:

• testlab user ownership in net
• admin ownership in subnet and higher

If I stay consistent with the testlab ownership, the loadbalancer still stays in the persistent state of 'ERROR'; however, it does not restrict me from deleting higher level objects that own it as I reached when the tenantry mismatched.

Am in the process of determining whether or not I can recreate the scenario of a locked loadbalancer creation when there's a tenant mismatch present on the system for a valid, consistent tenant's configuration.

[ssorenso]

So one last test I performed was under the conditions:

• Subnet with a different tenant than the one that owns the network
• Loadbalancer with the same tenant
• Listener with the same tenant

This resulted in the same issue. This was with a validly large subnet of 5 or so IP's: subnet-create --allocation-pool start=10.33.32.2,end=10.33.32.5 --gateway 10.33.32.1 --name test_net testlab-client-network 10.33.32.0/29 neutron lbaas-loadbalancer-create --name tstb test-net neutron lbaas-listener-create --name test_listener --loadbalancer tstb --protocol HTTPS --protocol-port 8080 //problems//

Example on that first bullet: User A owns the network. User B creates a subnet on that network. Continue.

[richbrowne]

Marked as duplicate. Added issue 402 in the driver repository since this is where the fix should go.

[richbrowne]

Fixed in driver