search

squat / generic-device-plugin

A Kubernetes device plugin to schedule generic Linux devices
Apache License 2.0
210 stars 23 forks source link

Generic Device Plugin vs KubeEdge #57

Closed zbm-armada closed 11 months ago

zbm-armada commented 1 year ago

Hi I'm looking to use a bluetooth usb dongle with my kubernetes cluster to connect with another bluetooth-enabled sensor.

I'm trying to understand the difference between this Generic Device Plugin and the device implementation in KubeEdge.

I'm curious how you all differentiate between the two. What are the tradeoffs between the this system and KubeEdge, is there one that is more secure/robust?

Thanks in advance.

squat commented 1 year ago

Hi @zbm-armada, I've never used KubeEdge, however my understanding of their Device CRD is that it fulfills pretty different functionality from the generic-device-plugin.

KubeEdge has CRDs that allow the administrator to define how to connect to and interact with a device that is accessible from an edge node and to declare the desired state of the device. KubeEdge then takes care of connecting to the device, e.g. over Bluetooth, and applying the state that the administrator declared. KubeEdge can report the status of the device by reading properties that the administrator has declared in the device CRD.

The generic-device-plugin is not concerned with managing the state of your devices at all. The plugin is concerned with allowing Kubernetes to schedule your devices so that they can be leveraged by your applications running at the edge. The plugin enables accessing your devices and the data produced by them by mounting them into your Pods, where processing of the data can take place.

IMO these two use-cases are distinct and potentially complimentary. I think it's worth noting that the two projects have somewhat disjoint definitions of what constitutes a device:

In other words, my reading is that KubeEdge allows you to define the parameters for connecting to a remote Bluetooth device from your edge node, whereas the generic-device-plugin allows you to allocate exclusive access to the Bluetooth dongle to one of your Pods so that the application running in the Pod can connect to and manage remote Bluetooth devices.

Re: security I don't know anything about how KubeEdge does security; I imagine they likely need to run privileged containers on the edge nodes, but that's just a guess.

generic-device-plugin allows Kubernetes to mount devices into your Pods so that you can access the device without privileged.

zbm-armada commented 1 year ago

Really appreciate you taking the time to respond, thank you.

I hear what you are saying about distinct and potentially complimentary. Sounds like generic-device-plugin has a similar approach to k3s, "Things can get tricky when the edge device is separate from the node, such as a remote sensor device that communicates to an edge node via Bluetooth; the K3S architecture will have a hard time handling this situation out of the box. On the other hand, KubeEdge is designed to work with edge nodes that do not have sensors, cameras and other types of edge devices attached directly to the motherboard." <-- This was the article that sparked my curiosity and lead me to finding generic-device-plugin

I guess it comes down to controlling the bluetooth connections entirely in the pod (generic-device-plugin), or defining the parameters for connection and letting KubeEdge handle the connections. The number of physically attached devices could help guide that decision - lots of bluetooth devices, go with KubeEdge. Physical dongle with just one bluetooth sensor connected? Go with generic-device-plugin.

Anyway I'm just trying to sort out my thoughts and understand the IoT Kubernetes landscape. Thanks again for your input!

squat commented 11 months ago

I think your overall analysis is spot on! I'll close this issue for now since it seems you know what direction to take. Please reopen if you need any more input.