search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
984 stars 76 forks source link

Disconnected diskless DRBD resources shown in LINSTOR as connected #342

Closed kvaps closed 1 year ago

kvaps commented 1 year ago 
root@k8s-prod-dc02-system-1:/# linstor v l -r pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ Node                   ┊ Resource                                 ┊ StoragePool          ┊ VolNr ┊ MinorNr ┊ DeviceName    ┊ Allocated ┊ InUse  ┊    State ┊
╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ k8s-prod-dc02-sts-1    ┊ pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 ┊ data                 ┊     0 ┊    1003 ┊ /dev/drbd1003 ┊  1.00 GiB ┊ Unused ┊ UpToDate ┊
┊ k8s-prod-dc02-sts-2    ┊ pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 ┊ data                 ┊     0 ┊    1003 ┊ /dev/drbd1003 ┊  1.00 GiB ┊ Unused ┊ UpToDate ┊
┊ k8s-prod-dc02-system-0 ┊ pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 ┊ DfltDisklessStorPool ┊     0 ┊    1003 ┊ /dev/drbd1003 ┊           ┊ InUse  ┊ Diskless ┊
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
root@k8s-prod-dc02-system-1:/# linstor r l -r pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ ResourceName                             ┊ Node                   ┊ Port ┊ Usage  ┊ Conns ┊    State ┊ CreatedOn           ┊
╞════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 ┊ k8s-prod-dc02-sts-1    ┊ 7003 ┊ Unused ┊ Ok    ┊ UpToDate ┊ 2023-02-06 03:54:07 ┊
┊ pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 ┊ k8s-prod-dc02-sts-2    ┊ 7003 ┊ Unused ┊ Ok    ┊ UpToDate ┊ 2023-02-06 03:54:05 ┊
┊ pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 ┊ k8s-prod-dc02-system-0 ┊ 7003 ┊ InUse  ┊ Ok    ┊ Diskless ┊ 2023-03-13 14:47:10 ┊
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
root@k8s-prod-dc02-system-1:/# drbdsetup status pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973
pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 role:Primary suspended:no-data,quorum
  disk:Diskless quorum:no blocked:upper
  k8s-prod-dc02-sts-1 connection:Connecting
  k8s-prod-dc02-sts-2 connection:Connecting
root@k8s-prod-dc02-sts-1:/# drbdsetup status pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973
pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 role:Secondary
  disk:UpToDate
  k8s-prod-dc02-sts-2 role:Secondary
    peer-disk:UpToDate
  k8s-prod-dc02-system-0 role:Primary
    peer-disk:Diskless
root@k8s-prod-dc02-sts-2:/# drbdsetup status pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973
pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 role:Secondary
  disk:UpToDate
  k8s-prod-dc02-sts-1 role:Secondary
    peer-disk:UpToDate
  k8s-prod-dc02-system-0 role:Primary
    peer-disk:Diskless

verbose:

root@k8s-prod-dc02-system-1:/# drbdsetup status pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 --verbose --statistics
pvc-e13b15bd-f6d5-4eab-b35c-9b534571f973 node-id:3 role:Primary
    suspended:no-data,quorum force-io-failures:no
    write-ordering:none
  volume:0 minor:1003 disk:Diskless client:yes backing_dev:none quorum:no
       size:1052408 read:0 written:0 al-writes:0 bm-writes:0 upper-pending:10 lower-pending:0
      al-suspended:no blocked:upper
  k8s-prod-dc02-sts-1 node-id:1 connection:Connecting role:Unknown congested:no
      ap-in-flight:88 rs-in-flight:0
    volume:0 replication:Off peer-disk:DUnknown resync-suspended:no
        received:0 sent:0 out-of-sync:0 pending:10 unacked:0
  k8s-prod-dc02-sts-2 node-id:0 connection:Connecting role:Unknown congested:no
      ap-in-flight:72 rs-in-flight:0
    volume:0 replication:Off peer-disk:DUnknown resync-suspended:no
        received:0 sent:0 out-of-sync:0 pending:9 unacked:0
kvaps commented 1 year ago

Ah, haven't found that k8s-prod-dc02-system-1 and k8s-prod-dc02-system-0 are different nodes. Seems duplicate of https://github.com/LINBIT/linstor-server/issues/287