search

ceph / ceph-nvmeof

Service to provide Ceph storage over NVMe-oF/TCP protocol
GNU Lesser General Public License v3.0
72 stars 39 forks source link

RBD image namespace resize doesn't reflect across cluster nodes for nvme subsystem #712

Open skbeura opened 3 weeks ago

skbeura commented 3 weeks ago

Have expanded a nvme namespace successfully on a 3-node ceph cluster. The new size was showing only on Node-01 and shows old size for other nodes while listing. If I expand the RBD image and same doesn't reflect on the nvme namespace. Is it not supported to reflect for the namespace? Pls let me know. Thanks.

Ceph Version: ceph version 18.2.2 (531c0d11a1c5d39fbfe6aa8a521f023abf3bf3e2) reef (stable) Host OS: Oracle Linux Server release 8.9 nvmeof & nvmeof-cli: v1.0.0 Protocol: NVMe over TCP

Initiator: VMware ESXi, 7.0.3

attached file for reference. nvme namespace resize issue.docx

gbregman commented 2 weeks ago

This seems like an SPDK issue. When an image is resized on a different node, SPDK doesn't update the size information for that image. So, when we read this info from the local SPDK we get old information. We can try replacing the usage of SPDK with RBD calls, to get the size.

skbeura commented 2 weeks ago

There is another issue that SPDK doesn't report the extended size normally where as RBD reports it correct. Below is one of the example where SPDK reports the new extended size for the nvme-image01 as 1.5PiB and RBD displays it correctly as 1.5TiB.

image
gbregman commented 2 weeks ago

@skbeura , this issue was already fixed, see here