search

ceph / ceph-nvmeof

Service to provide Ceph storage over NVMe-oF/TCP protocol
GNU Lesser General Public License v3.0
85 stars 44 forks source link

Device path status in VMware turns into Standby when ana-reporting & enable-ha are enabled on a subsystem #683

Open swariri opened 4 months ago

swariri commented 4 months ago

We have exposed a 4 TB namespace to VMware client using NVME TCP protocol from a Ceph cluster. Setup works fine till the time ana-reporting & enable-ha are disabled on a subsystem. But when those are enabled on a ceph cluster to achieve multipathing, device path status turns into standby in VMware.

Setup details - ceph version - 18.2.2 (531c0d11a1c5d39fbfe6aa8a521f023abf3bf3e2) reef (stable) Protocol- NVME TCP Hypervisor: VMware ESXi, 7.0.3, 23307199 nvmeof & nvmeof-cli version - 1.0.0

Is TCP multipathing supported on a Ceph cluster ?

caroav commented 4 months ago

The HA feature is not yet merged upstream on Ceph side. Hopefully it is getting very close. See this PR - https://github.com/ceph/ceph/pull/54671. If you want to test HA, you must build Ceph that includes this PR. And you should let the Ceph nvmeof monitor define ANA states.

swariri commented 4 months ago

Thanks for the reply. This would be help to pause further troubleshooting.