search

F5Networks / f5-openstack-agent

The F5 Agent for OpenStack Neutron allows you to deploy BIG-IP services in an OpenStack environment.
http://clouddocs.f5.com/products/openstack/agent/latest
Apache License 2.0
14 stars 38 forks source link

Queued Member Updates Result in Invalid Pools #709

Open jgruber opened 7 years ago

jgruber commented 7 years ago

Title: Creation of 1 pool with 20 members each on a BIG-IP leads to failures. Attachments: None - 100% reproducible with various log entries per run. I will post a clean logs on Monday. Details: Creation of 1 pool with 20 members on a BIG-IP pair results in the creation of 1 pool with far less then 20 pool members. This creates a difference in the LBaaS model in Neutron verse the BIG-IP configuration. Deletion of the pool members and pool leads to stranded Node Addresses and a tenant partition which does not have any LBaaS resources assigned in Neutron.

Agent Version

9.3.0

Operating System

CentOS

OpenStack Release

Mitaka

Deployment

Deploy LBaaS with a pair of BIG-IPs.

Use the following Heat template to orchestrate the 10 pool deployment, with input parameters adjusted to your cloud.

http://repo.mydemo.rocks/templates/labs/deploy_lbaas_20_web_servers_1_pools.yaml

You will need a web server glance image, and 2 subnets defined (one for the load balancer) and another for all nova server deployments.

Monitor the BIG-IP for pool members for difference between Neutron and BIG-IP once the neutron agent for the BIG-IP has reached a request depth of zero (thinks it is done provisioning).

Note the lack of pool members also results in the inability to delete the Heat stack properly as the driver will not allow the OS::Neutron::LBaaS::PoolMember to delete the member without an error from the driver.

Stranded Node Addresses on the BIG-IP are observed after pool members are deleted manually.

100% reproducible.

jgruber commented 7 years ago

I just ran 10 test of 20 pool members in 1 Pool with the 9.3.1 beta. It works after 8 mins per run. The only errors I got were delays in removing listeners before the community Heat resource tried to delete the loadbalancer. That's not our issue.

I will close this when I run the 10 pool 2 member Heat stack 10 times clean.

jgruber commented 7 years ago

Heat deployment with 10 pools of 2 members ran to completion 10 times.

However, standed Node Address, SNAT Pool, SNAT Translation Address, VxLAN tunnel, FDB record.

Neutron shows all LBaaS object delete, but Neutron ports for SNAT and Self IP stranded.

jgruber commented 7 years ago

There were no errors in the f5-openstack-agent log for deleting self-IPs, node addresses, or fdb records.

pjbreaux commented 7 years ago

Yo @jgruber: as far as I know, the lb provisioning status regulates the updates to the agent, meaning you cannot create a new member while a previous member is still in PENDING_UPDATE provisioning status. So I was wondering how you are creating these members?