search

canonical / charm-openstack-service-checks

Collection of Nagios checks and other utilities that can be used to verify the operation of an OpenStack cluster
0 stars 2 forks source link

openstack-service-check incorrectly reports OVN metadata port as DOWN #146

Closed sudeephb closed 6 months ago

sudeephb commented 6 months ago

According to [1], for each network, an OVN localport for use as a metadata proxy is created. This port is owned by network:dhcp (in Ussuri) or network:distributed (in Yoga) so that it gets auto deleted upon the removal of the network and it remains DOWN and is not bound to any chassis.

openstack-service-checks discovers that ports are DOWN with a "check_ports" check, and triggers Nagios alerts, e.g. "CRITICAL: ports 3/11 are DOWN, 1/11 in UNKNOWN, 7/11 passed". If I understand correctly, these ports are DOWN by design and therefore openstack-service-checks should not report it as failure.

See the commands' results below.

  1. https://docs.openstack.org/networking-ovn/latest/contributor/design/metadata_api.html#metadata-proxy-management-logic
ubuntu@sdtpdc41s100001:~/deploy$ openstack port list --device-owner network:distributed --long --format yaml
- Device Owner: network:distributed
  Fixed IP Addresses:
  - ip_address: fc00:3d46:c1c5:8992:f816:3eff:fef7:db16
    subnet_id: e2cac8d2-f773-4792-836a-3cbbfd8c9a8e
  ID: 2a95c95d-9049-4101-9eb5-9398843ab8e1
  MAC Address: fa:16:3e:f7:db:16
  Name: ''
  Security Groups: null
  Status: DOWN
  Tags: []
- Device Owner: network:distributed
  Fixed IP Addresses: []
  ID: 44e3ac74-3d67-4451-8246-032f5afc880f
  MAC Address: fa:16:3e:a0:97:d1
  Name: ''
  Security Groups: null
  Status: DOWN
  Tags: []
- Device Owner: network:distributed
  Fixed IP Addresses:
  - ip_address: 192.168.1.2
    subnet_id: 586d1335-2eca-4385-b27d-969aedade766
  ID: 51d4a0bb-adc9-4275-aabc-83f380451dda
  MAC Address: fa:16:3e:92:7b:8a
  Name: ''
  Security Groups: null
  Status: DOWN
  Tags: []

ubuntu@sdtpdc41s100001:~/deploy$ openstack port show 51d4a0bb-adc9-4275-aabc-83f380451dda --format yaml
admin_state_up: true
allowed_address_pairs: []
binding_host_id: ''
binding_profile: {}
binding_vif_details: {}
binding_vif_type: unbound
binding_vnic_type: normal
created_at: '2022-10-31T07:05:04Z'
data_plane_status: null
description: ''
device_id: ovnmeta-47e110ca-4e33-4a18-8505-53edd34ae023
device_owner: network:distributed
device_profile: null
dns_assignment:
- fqdn: host-192-168-1-2.lightning.ndc-prod2.lan.
  hostname: host-192-168-1-2
  ip_address: 192.168.1.2
dns_domain: ''
dns_name: ''
extra_dhcp_opts: []
fixed_ips:
- ip_address: 192.168.1.2
  subnet_id: 586d1335-2eca-4385-b27d-969aedade766
id: 51d4a0bb-adc9-4275-aabc-83f380451dda
ip_allocation: immediate
mac_address: fa:16:3e:92:7b:8a
name: ''
network_id: 47e110ca-4e33-4a18-8505-53edd34ae023
numa_affinity_policy: null
port_security_enabled: false
project_id: efd20e7f792f49d6982b9de4dbbb72b6
propagate_uplink_status: null
qos_network_policy_id: null
qos_policy_id: null
resource_request: null
revision_number: 2
security_group_ids: []
status: DOWN
tags: []
trunk_details: null
updated_at: '2022-10-31T07:05:04Z'

ubuntu@sdtpdc41s100001:~/deploy$ openstack subnet list --name ubuntu-subnet --format yaml
- ID: 586d1335-2eca-4385-b27d-969aedade766
  Name: ubuntu-subnet
  Network: 47e110ca-4e33-4a18-8505-53edd34ae023
  Subnet: 192.168.1.0/24

ubuntu@sdtpdc41s100001:~/deploy$ openstack subnet show ubuntu-subnet --format yaml
allocation_pools:
- end: 192.168.1.254
  start: 192.168.1.2
cidr: 192.168.1.0/24
created_at: '2022-10-31T07:05:04Z'
description: ''
dns_nameservers: []
dns_publish_fixed_ip: null
enable_dhcp: true
gateway_ip: 192.168.1.1
host_routes: []
id: 586d1335-2eca-4385-b27d-969aedade766
ip_version: 4
ipv6_address_mode: null
ipv6_ra_mode: null
name: ubuntu-subnet
network_id: 47e110ca-4e33-4a18-8505-53edd34ae023
project_id: efd20e7f792f49d6982b9de4dbbb72b6
revision_number: 0
segment_id: null
service_types: []
subnetpool_id: null
tags: []
updated_at: '2022-10-31T07:05:04Z'

ubuntu@sdtpdc41s100001:~/deploy$ juju run -a nova-compute sudo ip netns
[...]
- Stdout: |
    ovnmeta-47e110ca-4e33-4a18-8505-53edd34ae023 (id: 0)
  UnitId: nova-compute/6
[...]

ubuntu@sdtpdc41s100001:~/deploy$ juju ssh nova-compute/6 sudo ip netns exec ovnmeta-47e110ca-4e33-4a18-8505-53edd34ae023 ip -o -4 a
1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
2: tap47e110ca-41    inet 192.168.1.2/24 brd 192.168.1.255 scope global tap47e110ca-41\       valid_lft forever preferred_lft forever
2: tap47e110ca-41    inet 169.254.169.254/32 brd 169.254.169.254 scope global tap47e110ca-41\       valid_lft forever preferred_lft forever

ubuntu@sdtpdc41s100001:~/deploy$ juju ssh nova-compute/6 sudo ip netns exec ovnmeta-47e110ca-4e33-4a18-8505-53edd34ae023 ip r
192.168.1.0/24 dev tap47e110ca-41 proto kernel scope link src 192.168.1.2

Imported from Launchpad using lp2gh.