search

opiproject / opi-api

Open Programmable Infrastructure API and Behavioral Model
Apache License 2.0
32 stars 40 forks source link

Storage: Multipath support #297

Closed benlwalker closed 1 year ago

benlwalker commented 1 year ago
benlwalker commented 1 year ago

Looks okay to me. Before I LGTM, can you please confir if my understanding below is correct? NVMfPath refers to NVMfController, and NVMfNamespace also refers to NVMfController

So with this, now a user can define multiple NVMfPath objects (one for each real path), towards a controller that is linked to a backend NVMfRemoteNamespace object

In NVMe-oF a controller is a network session, but with multi-path there is often a nesting of "controllers" which I wish the spec would formalize with terminology. But alas, it does not. In this case, each NVMfPath is a true NVMe-oF controller (a session) established between a host and a subsystem. These paths are aggregated into an object that behaves like a single NVMe-oF controller - that's what the NVMfController represents (the aggregated thing). An NVMfRemoteNamespace is a namespace - the thing you can read/write - not a controller.

glimchb commented 1 year ago

In this case, each NVMfPath is a true NVMe-oF controller (a session) established between a host and a subsystem. These paths are aggregated into an object that behaves like a single NVMe-oF controller - that's what the NVMfController represents (the aggregated thing). An NVMfRemoteNamespace is a namespace - the thing you can read/write - not a controller.

@benlwalker looks like there are no RPCs to create/delete NVMfRemoteNamespace... so why have it as an object ? when it is created ? I guess when you connect to remote system with many namespaces, you probably want to create them internally, and then just use them when you reference that object in midllend/frontend... right ?

benlwalker commented 1 year ago

@benlwalker looks like there are no RPCs to create/delete NVMfRemoteNamespace... so why have it as an object ? when it is created ? I guess when you connect to remote system with many namespaces, you probably want to create them internally, and then just use them when you reference that object in midllend/frontend... right ?

If you imagine this is like the Linux kernel, there are no nvme-cli commands to create /dev/nvme0n1 (ignoring format namespace). Instead, you create /dev/nvme0 by connecting to a subsystem and it discovers the namespaces. You then list the namespaces and can mount filesystems on them. This is exactly the same thing. You discover the namespaces, then list them.

They need to be individual objects because these are what the middle/front end reference when building a middle end bdev or frontend nvmf subsystem. A crypto middle end bdev, for example, would be built on top of one of these namespace objects. Again it's just like Linux (or SPDK) where you create an md raid or a dm-crypt volume on top of /dev/nvme0n1 (the namespace you discovered), not on top of /dev/nvme0 (the controller you created).

glimchb commented 1 year ago

They need to be individual objects because these are what the middle/front end reference when building a middle end bdev or frontend nvmf subsystem. A crypto middle end bdev, for example, would be built on top of one of these namespace objects. Again it's just like Linux (or SPDK) where you create an md raid or a dm-crypt volume on top of /dev/nvme0n1 (the namespace you discovered), not on top of /dev/nvme0 (the controller you created).

thanks, that is what I figured out as I was writing the question. perhaps similarity with /dev/nvme0n1 and /dev/nvme0 somewhere in the comments could help