search

openebs-archive / cstor-operators

Collection of OpenEBS cStor Data Engine Operators
https://openebs.io
Apache License 2.0
94 stars 69 forks source link

cstor `MountVolume.MountDevice failed waiting for pvc-uid,s CVC to be bound` in k8s `v1.28.2` #452

Closed nAnderYang closed 10 months ago

nAnderYang commented 11 months ago

how do fix it ? thanks!!

OS: RockyLinux 9.2 Kernel: 5.14.0-284.11.1.el9_2.x86_64 k8s: v1.28.2 cni: calico 3.26.1 cStor: 3.3.0

update: I changed the operating system to rockylinux 8.8 and the situation remains the same.

use storageclass the pod always ContainerCreating

$  kubectl get pod
NAME                      READY   STATUS              RESTARTS   AGE
rwonce-74c757d89c-sz7h4   0/1     ContainerCreating   0          11m
$ kubectl describe pod rwonce-74c757d89c-sz7h4|tail -20
  rwo:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  rwo
    ReadOnly:   false
  kube-api-access-2vf4h:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age                 From               Message
  ----     ------            ----                ----               -------
  Warning  FailedScheduling  12m                 default-scheduler  0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling..
  Normal   Scheduled         12m                 default-scheduler  Successfully assigned default/rwonce-74c757d89c-sz7h4 to node-03
  Warning  FailedMount       25s (x13 over 12m)  kubelet            MountVolume.MountDevice failed for volume "pvc-689b56d6-0962-48b1-8217-b247beac05be" : rpc error: code = Internal desc = Waiting for pvc-689b56d6-0962-48b1-8217-b247beac05be's CVC to be bound
$ kubectl get sc
kubectl get sc
NAME        PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
cstor-csi   cstor.csi.openebs.io   Delete          Immediate           true                   22m
cstor-nfs   openebs.io/nfsrwx      Delete          Immediate           false                  22m
$ kubectl get pod -n openebs
NAME                                                              READY   STATUS    RESTARTS      AGE
cspc-operator-956d679df-f95qw                                     1/1     Running   0             30m
cstor-storage-d2rd-5667fc99d5-sm79g                               3/3     Running   0             29m
cstor-storage-jf5s-676c464bc6-cdgsm                               3/3     Running   0             29m
cstor-storage-ldb6-86c9f9cbc5-n6v8b                               3/3     Running   0             29m
cvc-operator-7bf8bb8bff-pf5tf                                     1/1     Running   0             30m
openebs-cstor-admission-server-757cdc9b46-vjn85                   1/1     Running   0             30m
openebs-cstor-csi-controller-0                                    6/6     Running   0             30m
openebs-cstor-csi-node-tmbbb                                      2/2     Running   0             30m
openebs-cstor-csi-node-vf8gv                                      2/2     Running   0             30m
openebs-cstor-csi-node-wgtcm                                      2/2     Running   0             30m
openebs-ndm-7fkhm                                                 1/1     Running   0             30m
openebs-ndm-cluster-exporter-89cc85b58-sh564                      1/1     Running   0             30m
openebs-ndm-node-exporter-dww9z                                   1/1     Running   0             30m
openebs-ndm-node-exporter-rxfj7                                   1/1     Running   0             30m
openebs-ndm-node-exporter-wtfnk                                   1/1     Running   0             30m
openebs-ndm-operator-7c7f58bd5f-56stz                             1/1     Running   0             30m
openebs-ndm-ssw6b                                                 1/1     Running   0             30m
openebs-ndm-tpxlz                                                 1/1     Running   0             30m
openebs-nfs-provisioner-86db8b8749-4ppk4                          1/1     Running   0             24m
pvc-689b56d6-0962-48b1-8217-b247beac05be-target-57c657668-z59gw   3/3     Running   4 (17m ago)   17m
nAnderYang commented 11 months ago

the pvc pod log

$ kubectl logs -n openebs pvc-689b56d6-0962-48b1-8217-b247beac05be-target-57c657668-z59gw -c cstor-istgt|grep -v ^$
 * Starting enhanced syslogd rsyslogd
rsyslog startup failure, child did not respond within startup timeout (60 seconds)
   ...done.
Disabling dumping core
2023-10-13/01:47:19.330 main              :3117: m#-145676352.6      : istgt:0.5.20121028:11:12:33:Jul 13 2022: starting
2023-10-13/01:47:19.335 main              :3165: m#-145676352.6      : istgt:0.5.20121028:11:12:33:Jul 13 2022 read config mode:1 gcc-atomic
2023-10-13/01:47:54.767 istgt_reload      :2169: u#10.140080136054528: reload configuration#1 start [istgt:0.5.20121028:11:12:33:Jul 13 2022]
2023-10-13/01:47:54.768 istgt_pg_update   :2338: m#-145676352.6      : update PG1
2023-10-13/01:47:54.768 istgt_lu_disk_init: 932: u#10.140080136054528: lu_disk_init LU1 TargetName=iqn.2016-09.com.openebs.cstor:pvc-689b56d6-0962-48b1-8217-b247beac05be
2023-10-13/01:47:54.768 istgt_lu_disk_init:1348: u#10.140080136054528: LU1: LUN0 iqn.2016-09.com.openebs.cstor:pvc-689b56d6-0962-48b1-8217-b247beac05be: storage_online  [(null), 1GB.0MB, 2097152 blks of 512 bytes, phy:3/6  WCE rpm:7200] q:32 thr:6/0 [0.0, 0.0]
2023-10-13/01:47:54.768 istgt_lu_disk_init:1365: u#10.140080136054528: LU1: LUN0 serial:8bc45dae-2a3c-4d0b-8e14-a2b58877095f
2023-10-13/01:47:54.769 istgt_lu_reload_up:3348: u#10.140080136054528: add LU1: Name=iqn.2016-09.com.openebs.cstor:pvc-689b56d6-0962-48b1-8217-b247beac05be
2023-10-13/01:47:54.769 istgt_reload      :2204: u#10.140080136054528: reload configuration#1 end [istgt:0.5.20121028:11:12:33:Jul 13 2022]
2023-10-13/01:47:55.311 accept_mgmt_conns :3080: rm#-176163072.6     : Accepted connection on descriptor(10) (host=10.250.190.15 port=51710)
2023-10-13/01:47:55.313 update_replica_ent:1501: rm#-176163072.6     : replica(6656611061693416998) connected successfully from 10.250.190.15:3232 rep: 3
2023-10-13/01:47:55.400 update_trusty_repl: 949: rm#-176163072.6     : Sending {"replicaId":"6900884696979742589","replicaZvolGuid":"6656611061693416998","volumeName":"pvc-689b56d6-0962-48b1-8217-b247beac05be","replicationFactor":3,"consistencyFactor":2} data to volume-mgmt to make it persistent
2023-10-13/01:47:55.400 istgt_connect_unx : 269: rm#-176163072.6     : uctl_connect_unx failed to /var/run/volume_mgmt_sock (errno=2)
2023-10-13/01:47:55.400 update_volume_conf: 816: rm#-176163072.6     : failed to connect to /var/run/volume_mgmt_sock
2023-10-13/01:47:55.400 update_trusty_repl: 957: rm#-176163072.6     : failed to update replica(6900884696979742589:6656611061693416998) as trusty replica
2023-10-13/01:47:55.400 update_replica_ent:1562: rm#-176163072.6     : Failed to update known trusty list... disconnecting new replica(ip:10.250.190.15 port:3232 guid:6656611061693416998 replica_id: 6900884696979742589)
2023-10-13/01:47:55.400 handle_mgmt_conn_e:4069: rm#-176163072.6     : Replica(6900884696979742589:6656611061693416998) got disconnected from 10.250.190.15:3232 mgmt_fd:-1
2023-10-13/01:47:55.400 accept_mgmt_conns :3080: rm#-176163072.6     : Accepted connection on descriptor(10) (host=10.250.254.76 port=45184)
2023-10-13/01:47:55.401 accept_mgmt_conns :3080: rm#-176163072.6     : Accepted connection on descriptor(12) (host=10.250.184.6 port=46510)
2023-10-13/01:47:55.403 update_replica_ent:1501: rm#-176163072.6     : replica(12443379608397205362) connected successfully from 10.250.254.76:3232 rep: 3
2023-10-13/01:47:55.426 update_trusty_repl: 949: rm#-176163072.6     : Sending {"replicaId":"1404166938188251823","replicaZvolGuid":"12443379608397205362","volumeName":"pvc-689b56d6-0962-48b1-8217-b247beac05be","replicationFactor":3,"consistencyFactor":2} data to volume-mgmt to make it persistent
2023-10-13/01:47:55.426 istgt_connect_unx : 269: rm#-176163072.6     : uctl_connect_unx failed to /var/run/volume_mgmt_sock (errno=2)
2023-10-13/01:47:55.426 update_volume_conf: 816: rm#-176163072.6     : failed to connect to /var/run/volume_mgmt_sock
2023-10-13/01:47:55.426 update_trusty_repl: 957: rm#-176163072.6     : failed to update replica(1404166938188251823:12443379608397205362) as trusty replica
2023-10-13/01:47:55.426 update_replica_ent:1562: rm#-176163072.6     : Failed to update known trusty list... disconnecting new replica(ip:10.250.254.76 port:3232 guid:12443379608397205362 replica_id: 1404166938188251823)
2023-10-13/01:47:55.426 handle_mgmt_conn_e:4069: rm#-176163072.6     : Replica(1404166938188251823:12443379608397205362) got disconnected from 10.250.254.76:3232 mgmt_fd:-1
2023-10-13/01:47:55.426 update_replica_ent:1501: rm#-176163072.6     : replica(12553841762696034076) connected successfully from 10.250.184.6:3232 rep: 3
2023-10-13/01:47:55.448 update_trusty_repl: 949: rm#-176163072.6     : Sending {"replicaId":"10884720162679812445","replicaZvolGuid":"12553841762696034076","volumeName":"pvc-689b56d6-0962-48b1-8217-b247beac05be","replicationFactor":3,"consistencyFactor":2} data to volume-mgmt to make it persistent
2023-10-13/01:47:55.448 istgt_connect_unx : 269: rm#-176163072.6     : uctl_connect_unx failed to /var/run/volume_mgmt_sock (errno=2)
2023-10-13/01:47:55.448 update_volume_conf: 816: rm#-176163072.6     : failed to connect to /var/run/volume_mgmt_sock
2023-10-13/01:47:55.448 update_trusty_repl: 957: rm#-176163072.6     : failed to update replica(10884720162679812445:12553841762696034076) as trusty replica
2023-10-13/01:47:55.448 update_replica_ent:1562: rm#-176163072.6     : Failed to update known trusty list... disconnecting new replica(ip:10.250.184.6 port:3232 guid:12553841762696034076 replica_id: 10884720162679812445)
2023-10-13/01:47:55.448 handle_mgmt_conn_e:4069: rm#-176163072.6     : Replica(10884720162679812445:12553841762696034076) got disconnected from 10.250.184.6:3232 mgmt_fd:-1
2023-10-13/01:47:59.409 accept_mgmt_conns :3080: rm#-176163072.6     : Accepted connection on descriptor(10) (host=10.250.190.15 port=51712)
2023-10-13/01:47:59.411 update_replica_ent:1501: rm#-176163072.6     : replica(6656611061693416998) connected successfully from 10.250.190.15:3232 rep: 3
2023-10-13/01:47:59.417 update_trusty_repl: 949: rm#-176163072.6     : Sending {"replicaId":"6900884696979742589","replicaZvolGuid":"6656611061693416998","volumeName":"pvc-689b56d6-0962-48b1-8217-b247beac05be","replicationFactor":3,"consistencyFactor":2} data to volume-mgmt to make it persistent
2023-10-13/01:47:59.455 istgt_uctl_cmd_res: 639: u#16.140080022812416: volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) requested to resize 1Gi
2023-10-13/01:47:59.455 istgt_uctl_cmd_res: 665: u#16.140080022812416: volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) size is already equal to requested resize
2023-10-13/01:48:00.433 update_replica_ent:1653: rm#-176163072.6     : replica(6900884696979742589:6656611061693416998) connected to target with needs_update: 1 can_be_trusty: 1
2023-10-13/01:48:00.433 update_volstate   :1189: rm#-176163072.6     : Marking volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) not ready for IOs
2023-10-13/01:48:00.434 accept_mgmt_conns :3080: rm#-176163072.6     : Accepted connection on descriptor(16) (host=10.250.254.76 port=45188)
2023-10-13/01:48:00.434 accept_mgmt_conns :3080: rm#-176163072.6     : Accepted connection on descriptor(18) (host=10.250.184.6 port=46516)
2023-10-13/01:48:00.435 update_replica_ent:1501: rm#-176163072.6     : replica(12553841762696034076) connected successfully from 10.250.184.6:3232 rep: 3
2023-10-13/01:48:00.442 update_trusty_repl: 949: rm#-176163072.6     : Sending {"replicaId":"10884720162679812445","replicaZvolGuid":"12553841762696034076","volumeName":"pvc-689b56d6-0962-48b1-8217-b247beac05be","replicationFactor":3,"consistencyFactor":2} data to volume-mgmt to make it persistent
2023-10-13/01:48:01.457 update_replica_ent:1653: rm#-176163072.6     : replica(10884720162679812445:12553841762696034076) connected to target with needs_update: 1 can_be_trusty: 1
2023-10-13/01:48:01.457 update_volstate   :1186: rm#-176163072.6     : volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) is ready for IOs now.. io_seq(10) healthy_replica(0) degraded_replica(2)
2023-10-13/01:48:01.458 update_replica_ent:1501: rm#-176163072.6     : replica(12443379608397205362) connected successfully from 10.250.254.76:3232 rep: 3
2023-10-13/01:48:01.462 update_trusty_repl: 949: rm#-176163072.6     : Sending {"replicaId":"1404166938188251823","replicaZvolGuid":"12443379608397205362","volumeName":"pvc-689b56d6-0962-48b1-8217-b247beac05be","replicationFactor":3,"consistencyFactor":2} data to volume-mgmt to make it persistent
2023-10-13/01:48:02.477 update_replica_ent:1653: rm#-176163072.6     : replica(1404166938188251823:12443379608397205362) connected to target with needs_update: 1 can_be_trusty: 1
2023-10-13/01:48:02.477 update_volstate   :1186: rm#-176163072.6     : volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) is ready for IOs now.. io_seq(10) healthy_replica(0) degraded_replica(3)
2023-10-13/01:48:12.488 update_replica_sta:2862: rm#-176163072.6     : Replica(12443379608397205362) state:0 rebuild status:0
2023-10-13/01:48:12.488 trigger_rebuild   : 704: rm#-176163072.6     : Healthy count(0) degraded count(3) consistency factor(2) replication factor(3) desired replication factor(3)
2023-10-13/01:48:12.488 trigger_rebuild   : 711: rm#-176163072.6     : Mesh rebuild will be attempted on replica(12443379608397205362) state:0
2023-10-13/01:48:12.488 update_replica_sta:2862: rm#-176163072.6     : Replica(6656611061693416998) state:0 rebuild status:0
2023-10-13/01:48:12.488 trigger_rebuild   : 622: rm#-176163072.6     : Rebuild is already in progress on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) for replica (12443379608397205362)
2023-10-13/01:48:12.488 update_replica_sta:2862: rm#-176163072.6     : Replica(12553841762696034076) state:0 rebuild status:0
2023-10-13/01:48:12.488 trigger_rebuild   : 622: rm#-176163072.6     : Rebuild is already in progress on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) for replica (12443379608397205362)
2023-10-13/01:48:12.490 start_rebuild     : 539: rm#-176163072.6     : start_rebuild opcode sent for Replica(12443379608397205362)state:0
2023-10-13/01:48:12.490 handle_prepare_for:2622: rm#-176163072.6     : Rebuild triggered on Replica(1404166938188251823:12443379608397205362) state:0
2023-10-13/01:48:22.501 update_replica_sta:2862: rm#-176163072.6     : Replica(12553841762696034076) state:0 rebuild status:0
2023-10-13/01:48:22.501 trigger_rebuild   : 622: rm#-176163072.6     : Rebuild is already in progress on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) for replica (12443379608397205362)
2023-10-13/01:48:22.501 update_replica_sta:2862: rm#-176163072.6     : Replica(12443379608397205362) state:1 rebuild status:3
2023-10-13/01:48:22.501 update_replica_sta:2879: rm#-176163072.6     : Replica(12443379608397205362) (10.250.254.76:3232) mgmt_fd:16 state changed from degraded to healthy
2023-10-13/01:48:22.501 update_replica_sta:2972: rm#-176163072.6     : Replica(12443379608397205362) marked healthy, seting master_replica to NULL
2023-10-13/01:48:22.501 update_volstate   :1186: rm#-176163072.6     : volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) is ready for IOs now.. io_seq(10) healthy_replica(1) degraded_replica(2)
2023-10-13/01:48:22.501 trigger_rebuild   : 704: rm#-176163072.6     : Healthy count(1) degraded count(2) consistency factor(2) replication factor(3) desired replication factor(3)
2023-10-13/01:48:22.501 trigger_rebuild   : 711: rm#-176163072.6     : Normal rebuild will be attempted on replica(12553841762696034076) state:0
2023-10-13/01:48:22.503 start_rebuild     : 539: rm#-176163072.6     : start_rebuild opcode sent for Replica(12553841762696034076)state:0
2023-10-13/01:48:22.503 handle_prepare_for:2622: rm#-176163072.6     : Rebuild triggered on Replica(10884720162679812445:12553841762696034076) state:0
2023-10-13/01:48:22.524 update_replica_sta:2862: rm#-176163072.6     : Replica(6656611061693416998) state:0 rebuild status:0
2023-10-13/01:48:22.524 trigger_rebuild   : 622: rm#-176163072.6     : Rebuild is already in progress on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) for replica (12553841762696034076)
2023-10-13/01:48:32.536 update_replica_sta:2862: rm#-176163072.6     : Replica(6656611061693416998) state:0 rebuild status:0
2023-10-13/01:48:32.536 trigger_rebuild   : 622: rm#-176163072.6     : Rebuild is already in progress on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) for replica (12553841762696034076)
2023-10-13/01:48:32.537 update_replica_sta:2862: rm#-176163072.6     : Replica(12553841762696034076) state:1 rebuild status:3
2023-10-13/01:48:32.537 update_replica_sta:2879: rm#-176163072.6     : Replica(12553841762696034076) (10.250.184.6:3232) mgmt_fd:18 state changed from degraded to healthy
2023-10-13/01:48:32.537 update_replica_sta:2972: rm#-176163072.6     : Replica(12553841762696034076) marked healthy, seting master_replica to NULL
2023-10-13/01:48:32.537 update_volstate   :1186: rm#-176163072.6     : volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) is ready for IOs now.. io_seq(10) healthy_replica(2) degraded_replica(1)
2023-10-13/01:48:32.537 trigger_rebuild   : 704: rm#-176163072.6     : Healthy count(2) degraded count(1) consistency factor(2) replication factor(3) desired replication factor(3)
2023-10-13/01:48:32.537 trigger_rebuild   : 711: rm#-176163072.6     : Normal rebuild will be attempted on replica(6656611061693416998) state:0
2023-10-13/01:48:32.538 start_rebuild     : 539: rm#-176163072.6     : start_rebuild opcode sent for Replica(6656611061693416998)state:0
2023-10-13/01:48:32.538 handle_prepare_for:2622: rm#-176163072.6     : Rebuild triggered on Replica(6900884696979742589:6656611061693416998) state:0
2023-10-13/01:48:42.550 update_replica_sta:2862: rm#-176163072.6     : Replica(6656611061693416998) state:1 rebuild status:3
2023-10-13/01:48:42.551 update_replica_sta:2879: rm#-176163072.6     : Replica(6656611061693416998) (10.250.190.15:3232) mgmt_fd:10 state changed from degraded to healthy
2023-10-13/01:48:42.551 update_replica_sta:2972: rm#-176163072.6     : Replica(6656611061693416998) marked healthy, seting master_replica to NULL
2023-10-13/01:48:42.551 update_volstate   :1186: rm#-176163072.6     : volume(pvc-689b56d6-0962-48b1-8217-b247beac05be) is ready for IOs now.. io_seq(10) healthy_replica(3) degraded_replica(0)
2023-10-13/01:48:42.551 trigger_rebuild   : 629: rm#-176163072.6     : No downgraded in_quorum replicas on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be), rebuild will be attempted on non_quorum replicas
2023-10-13/01:48:42.551 trigger_rebuild   : 630: rm#-176163072.6     : Healthy count:(3) Degraded count:(0) Non-quorum-replica count:(0)
2023-10-13/01:48:42.551 trigger_rebuild   : 634: rm#-176163072.6     : No downgraded replicas on volume(pvc-689b56d6-0962-48b1-8217-b247beac05be), rebuild will not be attempted
$ kubectl logs -n openebs pvc-689b56d6-0962-48b1-8217-b247beac05be-target-57c657668-z59gw -c maya-volume-exporter
I1013 01:46:19.166623       1 command.go:118] Starting openebs-exporter ...
I1013 01:46:19.166679       1 command.go:121] Initialising openebs-exporter for the cstor
I1013 01:46:19.166826       1 command.go:162] Registered openebs exporter for cstor
I1013 01:46:19.166842       1 server.go:41] Starting http server....
nAnderYang commented 11 months ago

the pvc pod mgmt logs

$  kubectl logs -n openebs pvc-689b56d6-0962-48b1-8217-b247beac05be-target-57c657668-z59gw -c cstor-volume-mgmt|grep -v ^$
+ volume_manager_pid=8
+ trap _sigint INT
+ trap _sigterm SIGTERM
+ wait 8
+ /usr/local/bin/volume-manager start
I1013 01:47:54.600726       8 server.go:137] Waiting for namespace to be populated
I1013 01:47:54.600995       8 unixsock.go:81] Client sent: STATUS
I1013 01:47:54.603011       8 unixsock.go:81] Client sent: STATUS
I1013 01:47:54.603583       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:47:54.603598       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:47:54.603812       8 unixsock.go:47] Client got: STATUS Running:NORMAL MODE OF OPERATION
OK STATUS
I1013 01:47:54.603823       8 unixsock.go:53] Appending line to resp: STATUS Running:NORMAL MODE OF OPERATION
I1013 01:47:54.603848       8 unixsock.go:53] Appending line to resp: OK STATUS
I1013 01:47:54.603856       8 unixsock.go:61] Breaking out of loop for line: OK STATUS
I1013 01:47:54.603860       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 STATUS Running:NORMAL MODE OF OPERATION
 OK STATUS
]
W1013 01:47:54.603977       8 server.go:42] Invalid listen port. Using default port 7777
I1013 01:47:54.603984       8 server.go:49] Starting gRPC server on port : 7777
I1013 01:47:54.604454       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
STATUS Running:NORMAL MODE OF OPERATION
OK STATUS
I1013 01:47:54.604466       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:47:54.604491       8 unixsock.go:53] Appending line to resp: STATUS Running:NORMAL MODE OF OPERATION
I1013 01:47:54.604522       8 unixsock.go:53] Appending line to resp: OK STATUS
I1013 01:47:54.604528       8 unixsock.go:61] Breaking out of loop for line: OK STATUS
I1013 01:47:54.604532       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 STATUS Running:NORMAL MODE OF OPERATION
 OK STATUS
]
I1013 01:47:54.660058       8 start.go:190] CStorVolume CRD found
I1013 01:47:54.660508       8 controller.go:75] Creating event broadcaster
I1013 01:47:54.660727       8 controller.go:93] Setting up event handlers
I1013 01:47:54.660900       8 runner.go:37] Starting CStorVolume controller
I1013 01:47:54.660914       8 runner.go:40] Waiting for informer caches to sync
I1013 01:47:54.664370       8 controller.go:105] Add event received for cstorvolume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:47:54.761775       8 runner.go:44] Starting CStorVolume workers
I1013 01:47:54.761799       8 runner.go:50] Started CStorVolume workers
I1013 01:47:54.766896       8 volume.go:144] Done writing istgt.conf
I1013 01:47:54.767174       8 unixsock.go:81] Client sent: REFRESH
I1013 01:47:54.769523       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:47:54.769538       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:47:54.769786       8 unixsock.go:47] Client got: OK REFRESH
I1013 01:47:54.769802       8 unixsock.go:53] Appending line to resp: OK REFRESH
I1013 01:47:54.769886       8 unixsock.go:61] Breaking out of loop for line: OK REFRESH
I1013 01:47:54.769933       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 OK REFRESH
]
I1013 01:47:54.770005       8 volume.go:151] Creating Iscsi Volume Successful
2023-10-13T01:47:54.770Z        INFO    volume/volume.go:152            {"eventcode": "cstor.volume.target.create.success", "msg": "Successfully created CStor volume target", "rname": "pvc-689b56d6-0962-48b1-8217-b247beac05be"}
I1013 01:47:54.778415       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:47:57.600914       8 server.go:137] Waiting for namespace to be populated
I1013 01:47:57.600995       8 server.go:146] CstorVolume namespace openebs
I1013 01:47:57.601001       8 server.go:148] Starting unix domain server
I1013 01:47:59.433044       8 config.go:145] Successfully updated pvc-689b56d6-0962-48b1-8217-b247beac05be volume with following replication information replication fator: from 3 to 3, consistencyFactor from 2 to 2 and known replica list with replicaId 6900884696979742589 and GUID 6656611061693416998
I1013 01:47:59.433278       8 handler.go:711] No status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:47:59.433294       8 handler.go:697] CStor volume modify event for volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:47:59.433459       8 controller.go:127] cStorVolume Modify event : pvc-689b56d6-0962-48b1-8217-b247beac05be, 8bc45dae-2a3c-4d0b-8e14-a2b58877095f
I1013 01:47:59.452805       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:47:59.453406       8 unixsock.go:81] Client sent: RESIZE pvc-689b56d6-0962-48b1-8217-b247beac05be 1Gi 10 60
I1013 01:47:59.453525       8 event.go:282] Event(v1.ObjectReference{Kind:"CStorVolume", Namespace:"openebs", Name:"pvc-689b56d6-0962-48b1-8217-b247beac05be", UID:"8bc45dae-2a3c-4d0b-8e14-a2b58877095f", APIVersion:"cstor.openebs.io/v1", ResourceVersion:"74811", FieldPath:""}): type: 'Normal' reason: 'Updated' Updated resize conditions
I1013 01:47:59.455734       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
OK RESIZE
I1013 01:47:59.455773       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:47:59.455816       8 unixsock.go:53] Appending line to resp: OK RESIZE
I1013 01:47:59.455823       8 unixsock.go:61] Breaking out of loop for line: OK RESIZE
I1013 01:47:59.455828       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 OK RESIZE
]
I1013 01:47:59.456276       8 volume.go:275] Updated '/usr/local/etc/istgt/istgt.conf' file with capacity '  LUN0 Storage 1Gi 32K'
2023-10-13T01:47:59.456Z        INFO    volume/volume.go:276            {"eventcode": "cstor.volume.target.resize.success", "msg": "Successfully resized CStor volume target", "rname": "pvc-689b56d6-0962-48b1-8217-b247beac05be", "capacity": "1Gi"}
I1013 01:47:59.468591       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:47:59.469345       8 event.go:282] Event(v1.ObjectReference{Kind:"CStorVolume", Namespace:"openebs", Name:"pvc-689b56d6-0962-48b1-8217-b247beac05be", UID:"8bc45dae-2a3c-4d0b-8e14-a2b58877095f", APIVersion:"cstor.openebs.io/v1", ResourceVersion:"74812", FieldPath:""}): type: 'Normal' reason: 'Updated' successfully resized volume from 0 to 1Gi
I1013 01:48:00.456704       8 config.go:145] Successfully updated pvc-689b56d6-0962-48b1-8217-b247beac05be volume with following replication information replication fator: from 3 to 3, consistencyFactor from 2 to 2 and known replica list with replicaId 10884720162679812445 and GUID 12553841762696034076
I1013 01:48:00.458252       8 handler.go:711] No status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:00.458514       8 handler.go:697] CStor volume modify event for volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:00.458668       8 controller.go:127] cStorVolume Modify event : pvc-689b56d6-0962-48b1-8217-b247beac05be, 8bc45dae-2a3c-4d0b-8e14-a2b58877095f
I1013 01:48:00.469029       8 handler.go:295] Ignoring changes for volume pvc-689b56d6-0962-48b1-8217-b247beac05be for operation modify
I1013 01:48:00.469328       8 event.go:282] Event(v1.ObjectReference{Kind:"CStorVolume", Namespace:"openebs", Name:"pvc-689b56d6-0962-48b1-8217-b247beac05be", UID:"8bc45dae-2a3c-4d0b-8e14-a2b58877095f", APIVersion:"cstor.openebs.io/v1", ResourceVersion:"74816", FieldPath:""}): type: 'Warning' reason: 'FailUpdate' Ignoring changes on volume pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:01.476502       8 handler.go:711] No status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:01.476530       8 handler.go:697] CStor volume modify event for volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:01.476537       8 controller.go:127] cStorVolume Modify event : pvc-689b56d6-0962-48b1-8217-b247beac05be, 8bc45dae-2a3c-4d0b-8e14-a2b58877095f
I1013 01:48:01.476706       8 config.go:145] Successfully updated pvc-689b56d6-0962-48b1-8217-b247beac05be volume with following replication information replication fator: from 3 to 3, consistencyFactor from 2 to 2 and known replica list with replicaId 1404166938188251823 and GUID 12443379608397205362
I1013 01:48:01.485295       8 handler.go:295] Ignoring changes for volume pvc-689b56d6-0962-48b1-8217-b247beac05be for operation modify
I1013 01:48:01.486791       8 event.go:282] Event(v1.ObjectReference{Kind:"CStorVolume", Namespace:"openebs", Name:"pvc-689b56d6-0962-48b1-8217-b247beac05be", UID:"8bc45dae-2a3c-4d0b-8e14-a2b58877095f", APIVersion:"cstor.openebs.io/v1", ResourceVersion:"74822", FieldPath:""}): type: 'Warning' reason: 'FailUpdate' Ignoring changes on volume pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:24.690847       8 unixsock.go:81] Client sent: REPLICA
I1013 01:48:24.691631       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Degraded","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Degraded","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":24},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Degraded","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":23},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":22}]}]}
OK REPLICA
I1013 01:48:24.691759       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:48:24.691796       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Degraded","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Degraded","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":24},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Degraded","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":23},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":22}]}]}
I1013 01:48:24.691859       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:48:24.691893       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:48:24.691939       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Degraded","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Degraded","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":24},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Degraded","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":23},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":22}]}]}
 OK REPLICA
]
I1013 01:48:24.700955       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:48:54.675250       8 unixsock.go:81] Client sent: REPLICA
I1013 01:48:54.675649       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":54},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":53},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":52}]}]}
OK REPLICA
I1013 01:48:54.675666       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:48:54.675821       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":54},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":53},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":52}]}]}
I1013 01:48:54.675941       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:48:54.675958       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:48:54.676036       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":54},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":53},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":52}]}]}
 OK REPLICA
]
I1013 01:48:54.707483       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:49:24.675530       8 unixsock.go:81] Client sent: REPLICA
I1013 01:49:24.676140       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":84},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":83},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":82}]}]}
OK REPLICA
I1013 01:49:24.676158       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:49:24.676295       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":84},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":83},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":82}]}]}
I1013 01:49:24.676315       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:49:24.676423       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:49:24.676491       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":84},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":83},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":82}]}]}
 OK REPLICA
]
I1013 01:49:24.687310       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:49:54.675808       8 unixsock.go:81] Client sent: REPLICA
I1013 01:49:54.676502       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":114},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":113},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":112}]}]}
OK REPLICA
I1013 01:49:54.676715       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:49:54.676875       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":114},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":113},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":112}]}]}
I1013 01:49:54.676987       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:49:54.677120       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:49:54.677216       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":114},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":113},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":112}]}]}
 OK REPLICA
]
I1013 01:49:54.688400       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:50:24.679482       8 unixsock.go:81] Client sent: REPLICA
I1013 01:50:24.680033       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":144},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":143},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":142}]}]}
OK REPLICA
I1013 01:50:24.680149       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:50:24.680225       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":144},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":143},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":142}]}]}
I1013 01:50:24.680302       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:50:24.680364       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:50:24.680441       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":144},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":143},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":142}]}]}
 OK REPLICA
]
I1013 01:50:24.690838       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:50:54.675861       8 unixsock.go:81] Client sent: REPLICA
I1013 01:50:54.676504       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":174},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":173},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":172}]}]}
OK REPLICA
I1013 01:50:54.676739       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:50:54.676851       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":174},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":173},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":172}]}]}
I1013 01:50:54.676956       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:50:54.677034       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:50:54.677117       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":174},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":173},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":172}]}]}
 OK REPLICA
]
I1013 01:50:54.686664       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:51:24.676156       8 unixsock.go:81] Client sent: REPLICA
I1013 01:51:24.676634       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":204},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":203},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":202}]}]}
OK REPLICA
I1013 01:51:24.676778       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:51:24.676886       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":204},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":203},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":202}]}]}
I1013 01:51:24.676974       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:51:24.677069       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:51:24.677127       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":204},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":203},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":202}]}]}
 OK REPLICA
]
I1013 01:51:24.707580       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:51:54.677152       8 unixsock.go:81] Client sent: REPLICA
I1013 01:51:54.677843       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":234},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":233},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":232}]}]}
OK REPLICA
I1013 01:51:54.678057       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:51:54.678176       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":234},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":233},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":232}]}]}
I1013 01:51:54.678265       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:51:54.678352       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:51:54.678363       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":234},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":233},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":232}]}]}
 OK REPLICA
]
I1013 01:51:54.688283       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:52:24.676518       8 unixsock.go:81] Client sent: REPLICA
I1013 01:52:24.677196       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":264},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":263},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":262}]}]}
OK REPLICA
I1013 01:52:24.677400       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:52:24.677530       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":264},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":263},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":262}]}]}
I1013 01:52:24.677622       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:52:24.677720       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:52:24.677808       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":264},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":263},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":262}]}]}
 OK REPLICA
]
I1013 01:52:24.686218       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:52:54.676622       8 unixsock.go:81] Client sent: REPLICA
I1013 01:52:54.677060       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":294},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":293},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":292}]}]}
OK REPLICA
I1013 01:52:54.677202       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:52:54.677283       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":294},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":293},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":292}]}]}
I1013 01:52:54.677341       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:52:54.677424       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:52:54.677564       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":294},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":293},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":292}]}]}
 OK REPLICA
]
I1013 01:52:54.686177       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:53:24.676739       8 unixsock.go:81] Client sent: REPLICA
I1013 01:53:24.677168       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":324},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":323},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":322}]}]}
OK REPLICA
I1013 01:53:24.677291       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:53:24.677372       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":324},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":323},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":322}]}]}
I1013 01:53:24.677476       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:53:24.677557       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:53:24.677625       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":324},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":323},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":322}]}]}
 OK REPLICA
]
I1013 01:53:24.687455       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:53:54.676125       8 unixsock.go:81] Client sent: REPLICA
I1013 01:53:54.676461       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":354},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":353},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":352}]}]}
OK REPLICA
I1013 01:53:54.676504       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:53:54.676541       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":354},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":353},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":352}]}]}
I1013 01:53:54.676568       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:53:54.676586       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:53:54.676602       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":354},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":353},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":352}]}]}
 OK REPLICA
]
I1013 01:53:54.684884       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:54:24.677216       8 unixsock.go:81] Client sent: REPLICA
I1013 01:54:24.677618       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":384},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":383},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":382}]}]}
OK REPLICA
I1013 01:54:24.677728       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:54:24.677873       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":384},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":383},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":382}]}]}
I1013 01:54:24.677987       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:54:24.678093       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:54:24.678200       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":384},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":383},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":382}]}]}
 OK REPLICA
]
I1013 01:54:24.686931       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:54:54.676759       8 unixsock.go:81] Client sent: REPLICA
I1013 01:54:54.677365       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":414},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":413},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":412}]}]}
OK REPLICA
I1013 01:54:54.677514       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:54:54.677648       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":414},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":413},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":412}]}]}
I1013 01:54:54.677663       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:54:54.677739       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:54:54.677809       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":414},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":413},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":412}]}]}
 OK REPLICA
]
I1013 01:54:54.686483       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:55:24.676942       8 unixsock.go:81] Client sent: REPLICA
I1013 01:55:24.677466       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":444},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":443},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":442}]}]}
OK REPLICA
I1013 01:55:24.677628       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:55:24.677707       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":444},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":443},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":442}]}]}
I1013 01:55:24.677868       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:55:24.677998       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:55:24.678145       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":444},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":443},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":442}]}]}
 OK REPLICA
]
I1013 01:55:24.689622       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:55:54.677836       8 unixsock.go:81] Client sent: REPLICA
I1013 01:55:54.678287       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":474},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":473},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":472}]}]}
OK REPLICA
I1013 01:55:54.678308       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:55:54.678448       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":474},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":473},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":472}]}]}
I1013 01:55:54.678518       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:55:54.678607       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:55:54.678669       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":474},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":473},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":472}]}]}
 OK REPLICA
]
I1013 01:55:54.688630       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:56:24.679577       8 unixsock.go:81] Client sent: REPLICA
I1013 01:56:24.680140       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":504},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":503},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":502}]}]}
OK REPLICA
I1013 01:56:24.680436       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:56:24.680611       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":504},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":503},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":502}]}]}
I1013 01:56:24.680793       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:56:24.680955       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:56:24.681073       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":504},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":503},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":502}]}]}
 OK REPLICA
]
I1013 01:56:24.692084       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:56:54.680081       8 unixsock.go:81] Client sent: REPLICA
I1013 01:56:54.680315       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:56:54.680849       8 unixsock.go:47] Client got:
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":534},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":533},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":532}]}]}
OK REPLICA
I1013 01:56:54.681035       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:56:54.681124       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":534},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":533},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":532}]}]}
I1013 01:56:54.681226       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:56:54.681318       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:56:54.681406       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":534},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":533},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":532}]}]}
 OK REPLICA
]
I1013 01:56:54.690416       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:57:24.681857       8 unixsock.go:81] Client sent: REPLICA
I1013 01:57:24.682347       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":564},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":563},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":562}]}]}
OK REPLICA
I1013 01:57:24.682520       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:57:24.682658       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":564},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":563},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":562}]}]}
I1013 01:57:24.682814       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:57:24.682955       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:57:24.683101       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":564},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":563},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":562}]}]}
 OK REPLICA
]
I1013 01:57:24.702010       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:57:54.680507       8 unixsock.go:81] Client sent: REPLICA
I1013 01:57:54.680953       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":594},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":593},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":592}]}]}
OK REPLICA
I1013 01:57:54.681099       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:57:54.681183       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":594},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":593},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":592}]}]}
I1013 01:57:54.681242       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:57:54.681301       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:57:54.681342       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":594},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":593},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":592}]}]}
 OK REPLICA
]
I1013 01:57:54.691050       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:58:24.681146       8 unixsock.go:81] Client sent: REPLICA
I1013 01:58:24.681898       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":624},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":623},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":622}]}]}
OK REPLICA
I1013 01:58:24.681924       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:58:24.682215       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":624},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":623},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":622}]}]}
I1013 01:58:24.682341       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:58:24.682475       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:58:24.682489       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":624},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":623},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":622}]}]}
 OK REPLICA
]
I1013 01:58:24.692196       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:58:54.679777       8 unixsock.go:81] Client sent: REPLICA
I1013 01:58:54.680444       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":654},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":653},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":652}]}]}
OK REPLICA
I1013 01:58:54.680627       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:58:54.680726       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":654},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":653},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":652}]}]}
I1013 01:58:54.680800       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:58:54.680875       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:58:54.680932       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":654},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":653},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":652}]}]}
 OK REPLICA
]
I1013 01:58:54.691476       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:59:24.680095       8 unixsock.go:81] Client sent: REPLICA
I1013 01:59:24.680728       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":684},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":683},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":682}]}]}
OK REPLICA
I1013 01:59:24.680950       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:59:24.681147       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":684},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":683},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":682}]}]}
I1013 01:59:24.681300       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:59:24.681483       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:59:24.681580       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":684},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":683},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":682}]}]}
 OK REPLICA
]
I1013 01:59:24.690140       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 01:59:54.681165       8 unixsock.go:81] Client sent: REPLICA
I1013 01:59:54.681757       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":714},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":713},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":712}]}]}
OK REPLICA
I1013 01:59:54.681926       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 01:59:54.682067       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":714},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":713},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":712}]}]}
I1013 01:59:54.682247       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 01:59:54.682405       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 01:59:54.682498       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":714},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":713},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":712}]}]}
 OK REPLICA
]
I1013 01:59:54.694577       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:00:24.681592       8 unixsock.go:81] Client sent: REPLICA
I1013 02:00:24.682259       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":744},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":743},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":742}]}]}
OK REPLICA
I1013 02:00:24.682306       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:00:24.682327       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":744},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":743},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":742}]}]}
I1013 02:00:24.682346       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:00:24.682360       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:00:24.682368       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":744},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":743},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":742}]}]}
 OK REPLICA
]
I1013 02:00:24.711898       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:00:54.681002       8 unixsock.go:81] Client sent: REPLICA
I1013 02:00:54.681691       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":774},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":773},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":772}]}]}
OK REPLICA
I1013 02:00:54.681867       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:00:54.682031       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":774},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":773},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":772}]}]}
I1013 02:00:54.682142       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:00:54.682250       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:00:54.682343       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":774},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":773},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":772}]}]}
 OK REPLICA
]
I1013 02:00:54.692816       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:01:24.681107       8 unixsock.go:81] Client sent: REPLICA
I1013 02:01:24.681798       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":804},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":803},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":802}]}]}
OK REPLICA
I1013 02:01:24.682047       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:01:24.682206       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":804},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":803},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":802}]}]}
I1013 02:01:24.682404       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:01:24.682563       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:01:24.682714       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":804},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":803},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":802}]}]}
 OK REPLICA
]
I1013 02:01:24.692553       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:01:54.680772       8 unixsock.go:81] Client sent: REPLICA
I1013 02:01:54.681186       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":834},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":833},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":832}]}]}
OK REPLICA
I1013 02:01:54.681209       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:01:54.681310       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":834},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":833},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":832}]}]}
I1013 02:01:54.681365       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:01:54.681408       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:01:54.681437       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":834},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":833},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":832}]}]}
 OK REPLICA
]
I1013 02:01:54.690684       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:02:24.680709       8 unixsock.go:81] Client sent: REPLICA
I1013 02:02:24.681269       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":864},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":863},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":862}]}]}
OK REPLICA
I1013 02:02:24.681418       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:02:24.681518       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":864},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":863},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":862}]}]}
I1013 02:02:24.681619       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:02:24.681727       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:02:24.681846       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":864},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":863},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":862}]}]}
 OK REPLICA
]
I1013 02:02:24.690193       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:02:54.681034       8 unixsock.go:81] Client sent: REPLICA
I1013 02:02:54.681564       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":894},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":893},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":892}]}]}
OK REPLICA
I1013 02:02:54.681702       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:02:54.681785       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":894},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":893},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":892}]}]}
I1013 02:02:54.681861       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:02:54.681954       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:02:54.682010       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":894},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":893},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":892}]}]}
 OK REPLICA
]
I1013 02:02:54.691796       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:03:24.680685       8 unixsock.go:81] Client sent: REPLICA
I1013 02:03:24.681118       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":924},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":923},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":922}]}]}
OK REPLICA
I1013 02:03:24.681135       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:03:24.681147       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":924},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":923},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":922}]}]}
I1013 02:03:24.681244       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:03:24.681257       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:03:24.681261       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":924},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":923},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":922}]}]}
 OK REPLICA
]
I1013 02:03:24.692059       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:03:54.681372       8 unixsock.go:81] Client sent: REPLICA
I1013 02:03:54.681599       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:03:54.681697       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:03:54.681819       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":954},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":953},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":952}]}]}
OK REPLICA
I1013 02:03:54.681917       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":954},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":953},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":952}]}]}
I1013 02:03:54.682012       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:03:54.682116       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:03:54.682200       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":954},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":953},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":952}]}]}
 OK REPLICA
]
I1013 02:03:54.690457       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:04:24.681297       8 unixsock.go:81] Client sent: REPLICA
I1013 02:04:24.681689       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":984},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":983},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":982}]}]}
OK REPLICA
I1013 02:04:24.681719       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:04:24.681733       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":984},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":983},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":982}]}]}
I1013 02:04:24.681765       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:04:24.681779       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:04:24.681787       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":984},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":983},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":982}]}]}
 OK REPLICA
]
I1013 02:04:24.693524       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:04:54.681673       8 unixsock.go:81] Client sent: REPLICA
I1013 02:04:54.681902       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:04:54.682046       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:04:54.682178       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1014},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1013},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1012}]}]}
OK REPLICA
I1013 02:04:54.682317       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1014},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1013},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1012}]}]}
I1013 02:04:54.682427       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:04:54.682465       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:04:54.682519       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1014},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1013},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1012}]}]}
 OK REPLICA
]
I1013 02:04:54.691169       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:05:24.681587       8 unixsock.go:81] Client sent: REPLICA
I1013 02:05:24.681808       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:05:24.681904       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:05:24.682032       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1044},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1043},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1042}]}]}
OK REPLICA
I1013 02:05:24.682135       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1044},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1043},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1042}]}]}
I1013 02:05:24.682254       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:05:24.682406       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:05:24.682498       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1044},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1043},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1042}]}]}
 OK REPLICA
]
I1013 02:05:24.693171       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:05:54.684978       8 unixsock.go:81] Client sent: REPLICA
I1013 02:05:54.685566       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:05:54.685594       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:05:54.685687       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1074},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1073},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1072}]}]}
OK REPLICA
I1013 02:05:54.685705       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1074},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1073},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1072}]}]}
I1013 02:05:54.685717       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:05:54.685729       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:05:54.685736       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1074},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1073},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1072}]}]}
 OK REPLICA
]
I1013 02:05:54.699723       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:06:24.682466       8 unixsock.go:81] Client sent: REPLICA
I1013 02:06:24.682806       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:06:24.682830       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:06:24.683070       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1104},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1103},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1102}]}]}
OK REPLICA
I1013 02:06:24.683091       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1104},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1103},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1102}]}]}
I1013 02:06:24.683173       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:06:24.683189       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:06:24.683194       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1104},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1103},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1102}]}]}
 OK REPLICA
]
I1013 02:06:24.696668       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:06:54.681922       8 unixsock.go:81] Client sent: REPLICA
I1013 02:06:54.682354       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:06:54.682614       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:06:54.682779       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1134},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1133},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1132}]}]}
OK REPLICA
I1013 02:06:54.682917       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1134},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1133},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1132}]}]}
I1013 02:06:54.683027       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:06:54.683168       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:06:54.683258       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1134},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1133},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1132}]}]}
 OK REPLICA
]
I1013 02:06:54.693512       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:07:24.682733       8 unixsock.go:81] Client sent: REPLICA
I1013 02:07:24.683122       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1164},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1163},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1162}]}]}
OK REPLICA
I1013 02:07:24.683144       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:07:24.683153       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1164},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1163},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1162}]}]}
I1013 02:07:24.683284       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:07:24.683291       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:07:24.683296       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1164},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1163},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1162}]}]}
 OK REPLICA
]
I1013 02:07:24.696321       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:07:54.683431       8 unixsock.go:81] Client sent: REPLICA
I1013 02:07:54.684115       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:07:54.684312       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:07:54.684488       8 unixsock.go:47] Client got: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1194},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1193},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1192}]}]}
OK REPLICA
I1013 02:07:54.684670       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1194},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1193},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1192}]}]}
I1013 02:07:54.684826       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:07:54.685010       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:07:54.685148       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1194},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1193},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1192}]}]}
 OK REPLICA
]
I1013 02:07:54.695508       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
I1013 02:08:24.683293       8 unixsock.go:81] Client sent: REPLICA
I1013 02:08:24.683815       8 unixsock.go:47] Client got: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1224},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1223},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1222}]}]}
OK REPLICA
I1013 02:08:24.683917       8 unixsock.go:53] Appending line to resp: iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
I1013 02:08:24.684008       8 unixsock.go:53] Appending line to resp: REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1224},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1223},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1222}]}]}
I1013 02:08:24.684093       8 unixsock.go:53] Appending line to resp: OK REPLICA
I1013 02:08:24.684178       8 unixsock.go:61] Breaking out of loop for line: OK REPLICA
I1013 02:08:24.684204       8 unixsock.go:71] response : [iSCSI Target Controller version istgt:0.5.20121028:11:12:33:Jul 13 2022 on  from
 REPLICA  {"volumeStatus":[{"name":"pvc-689b56d6-0962-48b1-8217-b247beac05be","status":"Healthy","replicaStatus":[{"replicaId":"6656611061693416998","Address":"10.250.190.15","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1224},{"replicaId":"12553841762696034076","Address":"10.250.184.6","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1223},{"replicaId":"12443379608397205362","Address":"10.250.254.76","Mode":"Healthy","checkpointedIOSeq":"0","inflightRead":"0","inflightWrite":"0","inflightSync":"0","quorum":"1","upTime":1222}]}]}
 OK REPLICA
]
I1013 02:08:24.696108       8 handler.go:705] Only status changed for cstor volume : pvc-689b56d6-0962-48b1-8217-b247beac05be
nAnderYang commented 10 months ago

update:

i use helm update cstor version to v3.5.0 .the problem has be fix.