Port conficts will occur when multiple pods dispatched to the same node under hostnetwork.

[Saturnoul]

Hostnetwork needs to be enabled to utilize RDMA for high performance transmission. In such circumstance, there will be port conflicts when multiple pods dispatched to the same node for the following reasons:

• sshd of all workers will listen at port 22
• ssh will use default port 22 for connection

Can mpi-operator handles the port conflicts under hostnetwork?

[alculquicondor]

Why not have one pod per node when using hostNetwork?

It would be convoluted to dynamically generate a port for each pod and put that into the hostfile.

[alculquicondor]

FWIIW, we use port 2222 by default in the base image, so that you can use hostNetwork without the pod's ssh agent conflicting with the host's ssh agent https://github.com/kubeflow/mpi-operator/blob/6d713b48e3617c0bb2567fedca624787a6108818/Makefile#L22

[tenzen-y]

FYI: Instead of hostNetwork, you could use SRIO-V device plugin and multus CNI.

In my production, the way works well.