search

Azure / iotedge

The IoT Edge OSS project
MIT License
1.46k stars 460 forks source link

Bring back kubernetes operator support. #7350

Closed Clockwork-Muse closed 3 months ago

Clockwork-Muse commented 3 months ago

As single-binary kubernetes distributions are maturing (or even full-blown kubernetes operating systems, like Talos Linux), the ability to spin up the edge components in such an environment becomes more appealing.

This includes scenarios on smaller devices, such as using k3s on IoT devices, where you want additional messaging reliability guarantees that aren't possible with the basic iot-device usage.

(It's possible to re-implement iot-edge "manually" by spinning up your own on-device messaging broker, but at that point you may as well use the whole iot-edge package)

veyalla commented 3 months ago

There are currently no plans to enable IoT Edge to run on Kubernetes beyond the KubeVirt approach. If you're looking for Microsoft-supported, Kubernetes-native messaging platform, checkout Azure IoT Operations.

Clockwork-Muse commented 3 months ago

The problem with Azure IoT Operations is that it's for "server room" deployments, whereas IoT Edge also allows for small-device deployments. This is particularly evident with the requirement from AIoTO to use Azure Arc - quite apart from the per-instance charge for Arc (which could be prohibitive at scale), the larger problem is management/enrollment, where IoT Edge allows for things like certificate chains for identity. Additionally, Arc isn't supported on arm/arm64... and the devices I'm working with are Raspberry Pis/Jetson Nanos (not x86_64), which means I couldn't deploy AIoTO there anyways.

The fact that I'm running on resource-constrained devices also makes the KubeVirt approach unappealing, as opposed to running modules on the "host" kubernetes directly.

My point of view here is more that the existing iot-edge agent is effectively acting as a kubernetes controller (and actually acting as a container orchestrator), and so fitting into that pattern may provide additional benefits. Among other things, this may allow for the deprecation of the secrets daemon (in preference for native kubernetes secrets management/existing plugins). Moving away from host-native daemons may also widen the set of supported systems (or possibly render it irrelevant), at least from the runtime side.

veyalla commented 3 months ago

Thanks for the additional context. I'd encourage you to create a feature request at https://aka.ms/iotedge-suggestion for tracking purposes and to enable other customers to upvote.

bishal41 commented 3 months ago

@Clockwork-Muse closing this issue. As mentioned above, please create a feature request at https://aka.ms/iotedge-suggestion for tracking purposes and to enable other customers to upvote. Thanks!

Clockwork-Muse commented 3 months ago

Feedback feature request