search

k8snetworkplumbingwg / sriov-network-operator

Operator for provisioning and configuring SR-IOV CNI plugin and device plugin
Apache License 2.0
80 stars 109 forks source link

pod resource request for openshift.io/mlnxnics: '2' doesn't work #767

Open jeffreyyjp opened 2 weeks ago

jeffreyyjp commented 2 weeks ago

Hi, team:

I have a question about pod request about sriov device.

When I config the pod with 

resources:
      requests:
        openshift.io/mlnxnics: '2'
      limits:
        openshift.io/mlnxnics: '2'

I can only get one VF mlx5_5 in my pod, and there are only one newly attached network inferface net1

And below is my pod annotation:

Annotations:      k8s.v1.cni.cncf.io/network-status:
                    [{
                        "name": "cilium_etcd_ipam_online_10.32.240.0_21",
                        "interface": "eth0",
                        "ips": [
                            "10.32.245.19"
                        ],
                        "mac": "9a:a8:6f:2a:8d:4b",
                        "default": true,
                        "dns": {},
                        "gateway": [
                            "169.254.100.100"
                        ]
                    },{
                        "name": "sriov-network-operator/sriov-network1",
                        "interface": "net1",
                        "ips": [
                            "192.168.1.200"
                        ],
                        "mac": "5e:c8:70:96:f0:d5",
                        "mtu": 1500,
                        "dns": {},
                        "device-info": {
                            "type": "pci",
                            "version": "1.1.0",
                            "pci": {
                                "pci-address": "0000:47:00.1",
                                "rdma-device": "mlx5_5"
                            }
                        }
                    }]
                  k8s.v1.cni.cncf.io/networks: sriov-network-operator/sriov-network1
jeffreyyjp commented 2 weeks ago

@SchSeba Could you help me to check this? Thanks

SchSeba commented 1 week ago

That is because you only request one network

k8s.v1.cni.cncf.io/networks: sriov-network-operator/sriov-network1

to have two you can do

k8s.v1.cni.cncf.io/networks: sriov-network-operator/sriov-network1,sriov-network-operator/sriov-network1

Adding the resource in the pod spec is not mandatory if you let the sriov operator deploy the resource injector webhook pod

jeffreyyjp commented 1 week ago

@SchSeba So for usual use cases, i only need to config k8s.v1.cni.cncf.io/networks in my pod, and doesn't need to care about device plugin resource?

SchSeba commented 1 week ago

if you have the resource injector deployed on the cluster that is right also if you are using a newer k8s version its good to enable this feature gate resourceInjectorMatchCondition

https://github.com/k8snetworkplumbingwg/sriov-network-operator?tab=readme-ov-file#available-feature-gates