search

eclipse-hono / hono

Eclipse Hono™ Project
https://eclipse.dev/hono
Eclipse Public License 2.0
450 stars 137 forks source link

Turn telemetry/event/command messages into CloudEvents #2512

Open sophokles73 opened 3 years ago

sophokles73 commented 3 years ago

The CloudEvents spec describes a set of standard (meta data) properties and transport protocol bindings which are used to allow compatible applications to recognize and process such events in a standardized way.

It shouldn't be too hard to add the missing properties to the messages exchanged by Hono's Telemetry, Event and C&C APIs.

avgustinmm commented 2 years ago

What is the idea here? AFAIU CloudEvents defines contract between event producer and consumers via context attributes, e.g. source/id, time, content type, subject,. Hono is an intermediary (in terms of CloudEvents) - events goes through Hono but it doesn't know anything about event id, real event/telemetry time. It may or may not know about content type / data format, Could assume the source is device id but in some use cases it might need to be more specific / different string
So, I can't see how Hono would add proper context attributes on behalf of devices. FMPOV the context shall be specified by the event sources - i.e. the devices that produce them. What Hono could do is to act as Intermediary - i.e. to propagate context attributes.

sophokles73 commented 2 years ago

Hono is an intermediary (in terms of CloudEvents)

It will certainly play that role if a device already uploads a CloudEvent message. However, the spec also explicitly refers to external Producers in the Source definition:

The "source" is the context in which the occurrence happened. In a distributed system it might consist of multiple Producers. If a source is not aware of CloudEvents, an external producer creates the CloudEvent on behalf of the source.

So I can also imagine Hono's protocol adapters playing that role for devices that can/do not upload CloudEvents. In that case, Hono would need to set the corresponding context properties on behalf of the device, e.g. the source could be set to something like my-tenant/my-device-id/${uuid} and type could be set to org.eclipse.hono.event ...

In addition to that, we could also add support for devices actually providing these properties when uploading messages and then simply use these values in the CloudEvent being sent downstream by the adapters.

WDYT?

avgustinmm commented 2 years ago

External producers are perfect match for use cases like sensors producing events and (higher level) device that communicates with the backend. However, concept is applicable to Hono, I suppose. Hono could supply source (device), id (that might be a little tricky to generated), specversion, type (e.g. org.eclipse.hono.event/telemetry), time - which would be received time and content-type (if available). This, however, would put some overhead on the Hono runtime and event size so should be configurable, disabled by default. But the most important FMPOV is to support CloudEvents as intermediary for CloudEvents aware devices - i.e. t o forward correspondent attributes. This, I think, should always be done. I wonder if a more generic feature should be implemented in Hono - tenant level configuration containing which received headers/attributes to be forwarded with event/telemetry.

sophokles73 commented 2 years ago

This, however, would put some overhead on the Hono runtime and event size so should be configurable, disabled by default.

agreed

But the most important FMPOV is to support CloudEvents as intermediary for CloudEvents aware devices - i.e. t o forward correspondent attributes. This, I think, should always be done.

agreed, and for this we would need to extend the protocol adapters accordingly. In most cases we already have the means to receive (arbitrary) meta data from the device(s) so it should not be too hard to support this. With the CoAP adapter it would be a little tricky because the Options in CoAP are based on (registered) numbers instead of string names.