libopenstorage / openstorage

A multi-host clustered implementation of the open storage specification
Apache License 2.0
523 stars 118 forks source link

PWX-38030: Don’t fail if node osdconfig doesn’t exist #2456

Closed vivekpx closed 2 months ago

vivekpx commented 2 months ago

What this PR does / why we need it:
Don’t fail if node osdconfig doesn’t exist

Which issue(s) this PR fixes (optional)
Closes # PWX-38030

Testing Notes

Node in Decommissioned state and cluster coordinator rebooted

root@vpatidar-99-3:~# pxctl status 
Status: PX is operational
Telemetry: Disabled or Unhealthy
Metering: Disabled or Unhealthy
License: Trial (expires in 31 days)
Node ID: 3c602ac3-b966-43d3-9669-98a749e8f0d4
    IP: 10.13.9.254 
    Local Storage Pool: 1 pool
    POOL    IO_PRIORITY RAID_LEVEL  USABLE  USED    STATUS  ZONE    REGION
    0   HIGH        raid0       111 GiB 34 MiB  Online  default default
    Local Storage Devices: 1 device
    Device  Path        Media Type      Size        Last-Scan
    0:0 /dev/sdd    STORAGE_MEDIUM_SSD  128 GiB     11 Jul 24 12:17 UTC
    total           -           128 GiB
    Cache Devices:
     * No cache devices
    Metadata Device: 
    1   /dev/sdc    STORAGE_MEDIUM_SSD  64 GiB
Cluster Summary
    Cluster ID: local-px-int
    Cluster UUID: 3933ff96-5d65-4533-9c68-4c882b220185
    Scheduler: kubernetes
    Total Nodes: 4 node(s) with storage (2 online)
    IP      ID                  SchedulerNodeName   Auth        StorageNode Used        Capacity    Status      StorageStatus   Version                 Kernel          OS
    10.13.8.208 db0f3fff-8890-49c8-8b5b-50a818d13565    vpatidar-99-1       Disabled    Yes(PX-StoreV2) 0 B     0 B     Decommissioned  Down        3.2.0-dt-multitenancy-fc-91c4120    6.5.0-27-generic    Ubuntu 22.04.3 LTS
    10.13.10.6  c9ab9292-63c3-4506-8fac-2c7648c871b9    vpatidar-99-2       Disabled    Yes(PX-StoreV2) 34 MiB      111 GiB     Online      Up3.2.0-dt-multitenancy-fc-91c4120  6.5.0-27-generic    Ubuntu 22.04.3 LTS
    10.13.9.254 3c602ac3-b966-43d3-9669-98a749e8f0d4    vpatidar-99-3       Disabled    Yes(PX-StoreV2) 34 MiB      111 GiB     Online      Up (This node)  3.2.0-dt-multitenancy-fc-91c4120    6.5.0-27-generic    Ubuntu 22.04.3 LTS
    10.13.8.95  022d10ae-1839-43c2-82bd-4c1b5aceea9f    vpatidar-99-4       Disabled    Yes(PX-StoreV2) Unavailable Unavailable Offline     Down        3.2.0-dt-multitenancy-fc-91c4120    6.5.0-27-generic    Ubuntu 22.04.3 LTS
Global Storage Pool
    Total Used      :  102 MiB
    Total Capacity  :  333 GiB

Rebooted node came back up and Decommissioned node got cleaned up

root@vpatidar-99-3:~# pxctl status 
Status: PX is operational
Telemetry: Disabled or Unhealthy
Metering: Disabled or Unhealthy
License: Trial (expires in 31 days)
Node ID: 3c602ac3-b966-43d3-9669-98a749e8f0d4
    IP: 10.13.9.254 
    Local Storage Pool: 1 pool
    POOL    IO_PRIORITY RAID_LEVEL  USABLE  USED    STATUS  ZONE    REGION
    0   HIGH        raid0       111 GiB 34 MiB  Online  default default
    Local Storage Devices: 1 device
    Device  Path        Media Type      Size        Last-Scan
    0:0 /dev/sdd    STORAGE_MEDIUM_SSD  128 GiB     11 Jul 24 12:17 UTC
    total           -           128 GiB
    Cache Devices:
     * No cache devices
    Metadata Device: 
    1   /dev/sdc    STORAGE_MEDIUM_SSD  64 GiB
Cluster Summary
    Cluster ID: local-px-int
    Cluster UUID: 3933ff96-5d65-4533-9c68-4c882b220185
    Scheduler: kubernetes
    Total Nodes: 3 node(s) with storage (3 online)
    IP      ID                  SchedulerNodeName   Auth        StorageNode Used    Capacity    Status  StorageStatus   Version                 Kernel          OS
    10.13.10.6  c9ab9292-63c3-4506-8fac-2c7648c871b9    vpatidar-99-2       Disabled    Yes(PX-StoreV2) 34 MiB  111 GiB     Online  Up      3.2.0-dt-multitenancy-fc-91c4120    6.5.0-27-generic    Ubuntu 22.04.3 LTS
    10.13.9.254 3c602ac3-b966-43d3-9669-98a749e8f0d4    vpatidar-99-3       Disabled    Yes(PX-StoreV2) 34 MiB  111 GiB     Online  Up (This node)  3.2.0-dt-multitenancy-fc-91c4120    6.5.0-27-generic    Ubuntu 22.04.3 LTS
    10.13.8.95  022d10ae-1839-43c2-82bd-4c1b5aceea9f    vpatidar-99-4       Disabled    Yes(PX-StoreV2) 34 MiB  111 GiB     Online  Up      3.2.0-dt-multitenancy-fc-91c4120    6.5.0-27-generic    Ubuntu 22.04.3 LTS
Global Storage Pool
    Total Used      :  102 MiB
    Total Capacity  :  333 GiB

Special notes for your reviewer:
https://jenkins.pwx.dev.purestorage.com/job/Dev/job/Porx-05/550/

github-actions[bot] commented 2 months ago

This PR is stale because it has been in review for 3 days with no activity.