search

squat / generic-device-plugin

A Kubernetes device plugin to schedule generic Linux devices
Apache License 2.0
198 stars 22 forks source link

Plugin doesn`t mount my usb devices inside pod #65

Open CroccoRush opened 6 months ago

CroccoRush commented 6 months ago

Hello :)

I have the following problem: I want to access a USB camera inside my minikube pod. My env: Ubuntu 22.04, Minikube v1.32.0, Kubernetes v1.29.0

I am download generic-device-plugin.yaml to my computer and run kubectl apply -f=generic-device-plugin.yaml. My node configuration after this looks like this: 

Name:               minikube
Roles:              control-plane
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=minikube
                    kubernetes.io/os=linux
                    minikube.k8s.io/commit=8220a6eb95f0a4d75f7f2d7b14cef975f050512d
                    minikube.k8s.io/name=minikube
                    minikube.k8s.io/primary=true
                    minikube.k8s.io/updated_at=2024_04_04T11_09_57_0700
                    minikube.k8s.io/version=v1.32.0
                    node-role.kubernetes.io/control-plane=
                    node.kubernetes.io/exclude-from-external-load-balancers=
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Thu, 04 Apr 2024 11:09:55 +0300
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  minikube
  AcquireTime:     <unset>
  RenewTime:       Mon, 08 Apr 2024 15:35:00 +0300
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Mon, 08 Apr 2024 15:35:00 +0300   Thu, 04 Apr 2024 11:09:54 +0300   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Mon, 08 Apr 2024 15:35:00 +0300   Thu, 04 Apr 2024 11:09:54 +0300   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Mon, 08 Apr 2024 15:35:00 +0300   Thu, 04 Apr 2024 11:09:54 +0300   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Mon, 08 Apr 2024 15:35:00 +0300   Fri, 05 Apr 2024 18:49:09 +0300   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  192.168.49.2
  Hostname:    minikube
Capacity:
  cpu:                12
  ephemeral-storage:  59536404Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             16100168Ki
  pods:               110
  squat.ai/audio:     10
  squat.ai/capture:   1
  squat.ai/fuse:      10
  squat.ai/serial:    0
  squat.ai/video:     1
Allocatable:
  cpu:                12
  ephemeral-storage:  59536404Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             16100168Ki
  pods:               110
  squat.ai/audio:     10
  squat.ai/capture:   1
  squat.ai/fuse:      10
  squat.ai/serial:    0
  squat.ai/video:     1
System Info:
  Machine ID:                 cab3c46fd9344d3d98dd8dcafedee792
  System UUID:                b288c57c-b602-4e49-adf7-c88f90a0c79a
  Boot ID:                    009a4e8a-d144-4dcc-9bd8-a91534acbc50
  Kernel Version:             6.5.0-26-generic
  OS Image:                   Ubuntu 22.04.3 LTS
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  docker://24.0.7
  Kubelet Version:            v1.29.0
  Kube-Proxy Version:         v1.29.0
PodCIDR:                      10.244.0.0/24
PodCIDRs:                     10.244.0.0/24
Non-terminated Pods:          (17 in total)
  Namespace                   Name                                          CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                          ------------  ----------  ---------------  -------------  ---
  default                     udp-server-89b74f87d-h9m8q                    0 (0%)        0 (0%)      0 (0%)           0 (0%)         17h
  default                     udp-server-deployment-5c8d47786c-c4hjz        0 (0%)        0 (0%)      0 (0%)           0 (0%)         17h
  default                     udp-server-deployment-c4476f65b-hpgqb         0 (0%)        0 (0%)      0 (0%)           0 (0%)         18h
  default                     udp-stream-deployment-b88d78b44-ldbch         0 (0%)        0 (0%)      0 (0%)           0 (0%)         16h
  kafka                       kafka-broker-94d6ff58d-8xpkp                  0 (0%)        0 (0%)      0 (0%)           0 (0%)         2d23h
  kafka                       zookeeper-cd79b98b6-hrqkq                     0 (0%)        0 (0%)      0 (0%)           0 (0%)         3d23h
  kube-system                 coredns-76f75df574-8ld6b                      100m (0%)     0 (0%)      70Mi (0%)        170Mi (1%)     4d4h
  kube-system                 etcd-minikube                                 100m (0%)     0 (0%)      100Mi (0%)       0 (0%)         4d4h
  kube-system                 generic-device-plugin-p8ztd                   50m (0%)      50m (0%)    10Mi (0%)        20Mi (0%)      6s
  kube-system                 kube-apiserver-minikube                       250m (2%)     0 (0%)      0 (0%)           0 (0%)         4d4h
  kube-system                 kube-controller-manager-minikube              200m (1%)     0 (0%)      0 (0%)           0 (0%)         4d4h
  kube-system                 kube-proxy-t5n4r                              0 (0%)        0 (0%)      0 (0%)           0 (0%)         4d4h
  kube-system                 kube-scheduler-minikube                       100m (0%)     0 (0%)      0 (0%)           0 (0%)         4d4h
  kube-system                 metrics-server-7c66d45ddc-6mknq               100m (0%)     0 (0%)      200Mi (1%)       0 (0%)         3d23h
  kube-system                 storage-provisioner                           0 (0%)        0 (0%)      0 (0%)           0 (0%)         4d4h
  kubernetes-dashboard        dashboard-metrics-scraper-7fd5cb4ddc-dwqkw    0 (0%)        0 (0%)      0 (0%)           0 (0%)         3d23h
  kubernetes-dashboard        kubernetes-dashboard-8694d4445c-klqrq         0 (0%)        0 (0%)      0 (0%)           0 (0%)         3d23h
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                900m (7%)   50m (0%)
  memory             380Mi (2%)  190Mi (1%)
  ephemeral-storage  0 (0%)      0 (0%)
  hugepages-1Gi      0 (0%)      0 (0%)
  hugepages-2Mi      0 (0%)      0 (0%)
  squat.ai/audio     0           0
  squat.ai/capture   0           0
  squat.ai/fuse      0           0
  squat.ai/serial    0           0
  squat.ai/video     0           0
Events:              <none>

And squat.ai/serial: 0 looks bad. I'm trying add -paths: - path: /dev/bus/usb/*/*/* to a serial groups. After this I got: squat.ai/serial: 10, but my camera doesn't throw inside pod.

In the next step I try to connect the simple webcam and add -paths: - path: /dev/video* to a video groups. And that doesn't work either.

squat commented 6 months ago

Linux kernel / udev is smart and knows how to discover USB cameras and interact with them driverlessly and puts them all under /dev/video. You should not normally have to manually configure discovery for the camera using serial or USB device hierarchies.

This device plugin by default discovers all cameras under /dev/usb and places groups them under the name squat.ai/video. Indeed you can see that the plugin already discovered a camera for your node. Your node's description shows:

Allocatable:
...
  squat.ai/video:     1

In fact the example in the readme for this project does exactly the same thing and uses squat.ai/video to allocate a USB webcam for a pod: https://github.com/squat/generic-device-plugin?tab=readme-ov-file#overview

Please try using squat.ai/video in your pod and let me know if that works for you.