got all VF with port state 'DOWN'

Mellanox / k8s-rdma-sriov-dev-plugin

Kubernetes Rdma SRIOV device plugin

Apache License 2.0

109 stars 27 forks source link

got all VF with port state 'DOWN' #26

Open yh-xu opened 4 years ago

yh-xu commented 4 years ago

I deployed rdma-sriov-dev-plugin in SRIOV mode for IB. The VFs were created but when running ibstat, I got the result that only mlx5_0 had port state "Active", while all others ( mlx5_1~mlx5_10 for 10 VFs) had port state "DOWN" (as image below). and I have checked policy for all VFs is "Follow".

I'm using ubuntu16.04 , kubernetes v 1.15. What should I do to make all VF ports active? Would it be something wrong in SM? I have SM running on IB switch with virtualization turned on.

paravmellanox commented 4 years ago

@yh-xu My suspect is SM config given that physical link is up

yh-xu commented 4 years ago

@yh-xu My suspect is SM config given that physical link is up

Does device-plugin work with hardware SM? I was referring this document (https://community.mellanox.com/s/article/kubernetes-ipoib-ethernet-rdma-sr-iov-networking-with-connectx4-connectx5?tdsourcetag=s_pctim_aiomsg ) which mentioned "opensm virtualization config", but didn't tell how to deal with hardware subnet manager.