Question about dataServiceEndPoint setting in csi

[gxglls]

Suppose I have three AHV hosts, each host has two network cards, one 1Gbit network card is used as a management network (192.168.10.0/20) , and a 10Git network card used as a storage network (10.0.10.0/24).

My VMs only allocates a network card with the management network 192.168.10.0/20 by default. I hope that dataServiceEndPoint is set on the storage network to avoid IO traffic going to the management network card. Assuming the dataServiceEndPoint is 10.0.0.100, does the vm need to add another network card for the storage network to access the dataServiceEndPoint? If it needs to be added, is there a risk that the broadcast domain will be too large to affect the services of the storage network when the number of VMs is too large?

On the other hand, fileHost only supports the configuration of one ip address, but fileserver does not have VIP. How to ensure high availability?

very thankful！

[subodh01]

Yes, you need to add another network on the VM if you do not want storage traffic to share network with management network. Have you looked at Network Segmentation feature ? https://portal.nutanix.com/page/documents/details?targetId=Nutanix-Security-Guide-v6_1:wc-network-segmentation-service-specific-isolation-c.html With this you can separate your iscsi (Nutanix Volumes) traffic.

Regarding Nutanix Files, you can use dynamic file storageclass for HA.

[gxglls]

@subodh01 your answer is very helpful! thanks a lot.

I continued to read many nutanix related documents, and I have one last question: Is IO localization possible via csi + dataServiceEndPoint? In the description of https://portal.nutanix.com/page/documents/solutions/details?targetId=BP-2049-Nutanix-Volumes:iscsi-target-redirection.html, I did not see the cvm data ip returned by iscsi-redirector ensure be local, will this cause the local vm to access the cvm on other hosts, resulting in read io across the network

[subodh01]

iscsi traffic is always over network, so it does not matter which CVMs host the disk. You will get the same performance. If you use network segmentation and the VM also has a network interface on the segmented network, IO traffic will be all layer 2 i.e no routing is involved.

[gxglls]

The IO localization I described means that the vm and the cvm are on the same physical machine (https://www.nutanixbible.com/4c-book-of-aos-dsf.html#data-locality), so that the vm's read io is not need to span physical networks (lower overhead than "all layer 2")