search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
954 stars 76 forks source link

Inconsistent resources shown as SyncTarget #203

Closed kvaps closed 2 years ago

kvaps commented 3 years ago 
# linstor r l -r one-vm-7707-disk-0
╭────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ ResourceName       ┊ Node   ┊ Port ┊ Usage  ┊ Conns ┊              State ┊ CreatedOn           ┊
╞════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ one-vm-7707-disk-0 ┊ m10c41 ┊ 7737 ┊ Unused ┊ Ok    ┊           UpToDate ┊ 2020-12-04 11:37:33 ┊
┊ one-vm-7707-disk-0 ┊ m11c4  ┊ 7737 ┊ InUse  ┊ Ok    ┊           Diskless ┊ 2020-12-04 09:56:25 ┊
┊ one-vm-7707-disk-0 ┊ m15c44 ┊ 7737 ┊ Unused ┊ Ok    ┊ SyncTarget(97.93%) ┊ 2020-12-05 15:27:38 ┊
╰────────────────────────────────────────────────────────────────────────────────────────────────╯
root@m15c44:~# drbdadm status one-vm-7707-disk-0
one-vm-7707-disk-0 role:Secondary
  disk:Inconsistent
  m10c41 role:Secondary
    peer-disk:UpToDate
  m11c4 role:Primary
    peer-disk:Diskless

root@m15c44:~# drbdadm disconnect one-vm-7707-disk-0:^C
root@m15c44:~# ^C
root@m15c44:~# drbdsetup status --verbose --statistics one-vm-7707-disk-0
one-vm-7707-disk-0 node-id:1 role:Secondary suspended:no
    write-ordering:flush
  volume:0 minor:1737 disk:Inconsistent quorum:yes
      size:29362008 read:29361592 written:29096996 al-writes:363 bm-writes:0 upper-pending:0 lower-pending:0 al-suspended:no blocked:no
  m10c41 node-id:0 connection:Connected role:Secondary congested:no ap-in-flight:0 rs-in-flight:0
    volume:0 replication:Established peer-disk:UpToDate resync-suspended:no
        received:28271508 sent:0 out-of-sync:0 pending:0 unacked:0
  m11c4 node-id:2 connection:Connected role:Primary congested:no ap-in-flight:0 rs-in-flight:0
    volume:0 replication:Established peer-disk:Diskless peer-client:yes resync-suspended:no
        received:825488 sent:0 out-of-sync:0 pending:0 unacked:0
raltnoeder commented 3 years ago

That one should probably be a SyncTarget, the question is rather why m10c41 is not a SyncSource and m15c44 is not showing SyncTarget in the DRBD status

kvaps commented 3 years ago

Not sure, I guess some drbd bug, we're facing with this quite often on versions 9.0.21, 9.0.22, 9.0.23

Exactly this situation can be simple solved by

drbdadm disconnect one-vm-7707-disk-0:m10c41
drbdadm connect one-vm-7707-disk-0:m10c41
rp- commented 2 years ago

Seems more related to DRBD until we would miss some drbdevents2 stream data. Anyhow we would need a reproducer for that drbdevents2 issues