[Bug] [STACK-2027] thunder reload detection

[ytsai-a10]

Description

• Severity Level: High
• Description:

1. Detect thunder reload and mark ERROR for loadbalancers which has lost configration. (STACK-2027)

see also : https://github.com/a10networks/a10-octavia/pull/308

Jira Ticket

https://a10networks.atlassian.net/browse/STACK-2027

Technical Approach

STACK-2027

1. Following aXAPI can show a10lb up time, maybe we can use it to compare with update_at to find reloaded/rebooted thunders. /axapi/v3/miscellenious-alb/oper
2. After find reloaded thunders, mark loadbalancers status as ERROR.

Config Changes

[a10_house_keeping]
use_periodic_write_memory = 'enable'
write_mem_interval = 600

Test Cases

• Test reload thunders before write memory period

Manual Testing

• create 5 loadbalancers and wait write memory
• set 2 of them

  openstack loadbalancer set vip5
  openstack loadbalancer set vip3

• reload thunder and wait next write memory interval
• check configuration lost thunders will mark as ERROR

  openstack loadbalancer list
  +--------------------------------------+------+----------------------------------+---------------+---------------------+----------+
  | id                                   | name | project_id                       | vip_address   | provisioning_status | provider |
  +--------------------------------------+------+----------------------------------+---------------+---------------------+----------+
  | 862e83f9-7155-4a05-9de6-0c6e52e8385e | vip1 | 3d21c71c4e4040599933bda62a76ae2f | 192.168.91.56 | ACTIVE              | a10      |
  | bcb68a38-c60b-4343-b66a-23ead86a530c | vip2 | 3d21c71c4e4040599933bda62a76ae2f | 192.168.91.57 | ACTIVE              | a10      |
  | d39ed514-df83-496c-b725-80a19ed44b6d | vip3 | 3d21c71c4e4040599933bda62a76ae2f | 192.168.91.58 | ERROR               | a10      |
  | d3fd13fa-d637-4e60-aeae-4eedd4b797c0 | vip4 | 3d21c71c4e4040599933bda62a76ae2f | 192.168.91.59 | ACTIVE              | a10      |
  | ec52d21d-dd2b-41ea-91cf-a425b7265daa | vip5 | 3d21c71c4e4040599933bda62a76ae2f | 192.168.91.60 | ERROR               | a10      |
  +--------------------------------------+------+----------------------------------+---------------+---------------------+----------+