F5Networks / f5-openstack-lbaasv1

OpenStack Neutron LBaaSv1 plugin and agent to control F5 BIG-IP devices
http://f5-openstack-lbaasv1.readthedocs.io
6 stars 8 forks source link

Agent repeatedly changes autosync setting on BIG-IP #132

Closed mattgreene closed 7 years ago

mattgreene commented 7 years ago

Agent Version

1.0.14 (effective)

Operating System

N/A

OpenStack Release

Kilo

Description

F5 agent continuously attempts to disable the auto-sync setting on BIG-IP. It also appears that the agent stuck in a cycle failing to connect to the BIG-IP and restarting.

Deployment

N/A

Additional Information

This agent has attempted to disable auto-sync (different than the value configured in the agent ini file) 7173 time since 4AM today, even though it is already disabled. Here is a sample from the audit log today:  

Oct 26 09:25:58 ccg21b02appf5-1a-v1 notice mcpd[8496]: 01070417:5: AUDIT - client tmsh, tmsh-pid-20138, user lbaas - transaction #71920692-2 - object 0 - modify { devicegroup { devicegroup_name "/Common/failover-group" devicegroup_autosync_enabled 0 } } [Status=Command OK]
Oct 26 09:25:58 ccg21b02appf5-1a-v1 notice icrd_child[20138]: 01420002:5: AUDIT - pid=20138 user=lbaas folder=/Common module=(tmos)# status=[Command OK] cmd_data=modify cm device-group /Common/failover-group { auto-sync disabled }
Oct 26 09:26:03 ccg21b02appf5-1a-v1 notice mcpd[8496]: 01070417:5: AUDIT - client tmsh, tmsh-pid-20138, user lbaas - transaction #71920886-2 - object 0 - modify { devicegroup { devicegroup_name "/Common/failover-group" devicegroup_autosync_enabled 0 } } [Status=Command OK]
Oct 26 09:26:03 ccg21b02appf5-1a-v1 notice icrd_child[20138]: 01420002:5: AUDIT - pid=20138 user=lbaas folder=/Common module=(tmos)# status=[Command OK] cmd_data=modify cm device-group /Common/failover-group { auto-sync disabled }
mattgreene commented 7 years ago
2016-10-31 17:12:11,346 f5.oslbaasv1agent.drivers.bigip.icontrol_driver     INFO Opening iControl connection to lbaas @ <redact>
2016-10-31 17:12:13,132 requests.packages.urllib3.connectionpool     INFO Starting new HTTPS connection (1): <redact>
2016-10-31 17:12:13,157 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/sys/db/provision.extramb HTTP/1.1" 200 266
2016-10-31 17:12:13,272 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/sync-status?$select=status HTTP/1.1" 200 131
2016-10-31 17:12:13,279 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/device-group/?$select=name,type HTTP/1.1" 200 285
2016-10-31 17:12:13,287 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/device/?$select=name,selfDevice&filter%20partition%20eq%20Common HTTP/1.1" 200 292
2016-10-31 17:12:13,293 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/net/interface/?$select=macAddress HTTP/1.1" 200 320
2016-10-31 17:12:13,299 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/net/interface/?$select=name,macAddress HTTP/1.1" 200 393
2016-10-31 17:12:13,317 requests.packages.urllib3.connectionpool    DEBUG "PUT /mgmt/tm/cm/device-group/~Common~failover-group HTTP/1.1" 200 553
2016-10-31 17:12:13,325 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/sys/db/iptunnel.configsync HTTP/1.1" 200 273
2016-10-31 17:12:13,333 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/device-group/?$select=name,type HTTP/1.1" 200 285
2016-10-31 17:12:13,340 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/device-group/~Common~failover-group/devices?$select=name HTTP/1.1" 200 306
2016-10-31 17:12:13,351 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/device/?$select=managementIp,name HTTP/1.1" 200 315
2016-10-31 17:12:13,358 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/device/?$select=managementIp,name HTTP/1.1" 200 315
2016-10-31 17:12:13,364 requests.packages.urllib3.connectionpool    DEBUG "GET /mgmt/tm/cm/traffic-group?$select=name HTTP/1.1" 200 212
2016-10-31 17:12:13,365 f5.oslbaasv1agent.drivers.bigip.icontrol_driver     INFO Opening iControl connection to lbaas @ None
2016-10-31 17:12:13,367 f5.oslbaasv1agent.drivers.bigip.icontrol_driver    ERROR Could not communicate with all iControl devices: 
2016-10-31 17:12:18,371 neutron CRITICAL URLError: <urlopen error [Errno -2] Name or service not known>
Traceback (most recent call last):
  File "/opt/openstack/neutron/2015.1.1.r1319/bin/f5-oslbaasv1-agent", line 26, in <module>
    sys.exit(main())
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/agent.py", line 69, in main
    mgr = manager.LbaasAgentManager(cfg.CONF)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/agent_manager.py", line 959, in __init__
    LbaasAgentManagerBase.do_init(self, conf)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/agent_manager.py", line 223, in do_init
    conf.f5_bigip_lbaas_device_driver, self.conf)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/oslo_utils/importutils.py", line 38, in import_object
    return import_class(import_str)(*args, **kwargs)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/icontrol_driver.py", line 344, in __init__
    self.connect_bigips()
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/icontrol_driver.py", line 377, in connect_bigips
    self._init_bigips()
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/icontrol_driver.py", line 522, in _init_bigips
    bigip = self._open_bigip(hostname)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/oslbaasv1agent/drivers/bigip/icontrol_driver.py", line 545, in _open_bigip
    f5const.CONNECTION_TIMEOUT)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/bigip.py", line 60, in __init__
    self.icontrol = self._get_icontrol(hostname, username, password, timeout, self.iControlSoapInterfaces)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/bigip.py", line 343, in _get_icontrol
    wsdls=iControlSoapInterfaces)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/pycontrol/pycontrol.py", line 100, in __init__
    self.clients = self._get_clients()
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/pycontrol/pycontrol.py", line 179, in _get_clients
    client = self._get_client(wsdl)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/pycontrol/pycontrol.py", line 173, in _get_client
    return self._get_suds_client(url, **self.kw)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/pycontrol/pycontrol.py", line 232, in _get_suds_client
    password=self.password, doctor=DOCTOR, **kw)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/f5/bigip/pycontrol/pycontrol.py", line 359, in __init__
    self.wsdl = reader.open(url)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/reader.py", line 150, in open
    d = self.fn(url, self.options)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/wsdl.py", line 136, in __init__
    d = reader.open(url)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/reader.py", line 74, in open
    d = self.download(url)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/reader.py", line 92, in download
    fp = self.options.transport.open(Request(url))
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/transport/https.py", line 62, in open
    return HttpTransport.open(self, request)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/transport/http.py", line 67, in open
    return self.u2open(u2request)
  File "/opt/openstack/neutron/2015.1.1.r1319/lib/python2.7/site-packages/suds/transport/http.py", line 132, in u2open
    return url.open(u2request, timeout=tm)
  File "/usr/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1222, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open
    raise URLError(err)
URLError: <urlopen error [Errno -2] Name or service not known>
dflanigan commented 7 years ago

Matt completed work on this on Nov 18.