homieiot / convention

🏡 The Homie Convention: a lightweight MQTT convention for the IoT
https://homieiot.github.io/
Other
716 stars 61 forks source link

property attribute $interval #53

Closed ingoogni closed 6 years ago

ingoogni commented 6 years ago

In the past days, working with the Homie Convention, I on occasions thought, this [...] is missing, on deeper though it was not. But, there is always a but, I now ran into an "omission" that I think would be a useful addition to the convention:

homie / device ID / node ID / property / property attribute

EDIT: as intervals do not have to be seconds but can also be other units as meters or °C another property attribute may be needed: $intervalunit (see in discussion below)

Property attribute: $interval is an attribute of a property, not a property of a node (although a node could have). It tells us something about the data being published. My CO2 sensor measures every 5 seconds, I have the data published ever 10 seconds and a min, avg, max every 5 minutes.

Some use cases:

Monitoring; A client can be written that monitors the timely arrival of data packets and responds to the situation.

Time series; Even if it would be easy to add time stamps to data it is a bit clumsy to write time series to a database row by row by time stamp. Easier is it to write only a start and or end time and write the data to an array in the DB, that though needs a regular and known interval for the data. A published $interval is this and makes it possible to for a client to note missed packets and write null data in such an array. A round robin database can use $interval for setting the 'heartbeat'. A series Timestamps itself will not tell us whether they come in at the set or required interval.

State changes; During a process the state of a sensor can change and with it its publishing frequency. With a (re-)published interval clients can adapt.

Stale Data; As all messages are published retained a newly active client can determine by use of the $interval whether the first dataset received by the broker is stale and discard it in that case.

fermuch commented 6 years ago

I really like the idea of setting the intervals from the controller to the device, so the controller can ask for more data if it's necessary.

Example:

Normally, you only want to read the temperature of a room every minute or so. But if some sensor detects smoke, you'd like to switch the temperature sensors to a higher frequency, so you can detect where the fire is propagating. You could do this filtering on the server side, but if it's running over GPRS it's way better to ask more only when needed.

I do something similar to this with broadcasts. If there's a "critical" broadcast, all sensors switch to critical mode, sending as much data as they can, and when things go back to normal I fire a "normal" broadcast.

ingoogni commented 6 years ago

... and then the next morning I'm measuring the efficiency of my boiler, guess what, the interval is not in seconds but I measure seconds looking at a 1°C interval, being a property of the node.

bertmelis commented 6 years ago

I don't see the limitation of the (soon-to-be-final) Homie convention. Although there's a recommendation of the units, an "interval" or "time-base" attribute is allowed. You can even make it a settable attribute.

homie/686f6d6965/temperature/$type → temperature
homie/686f6d6965/temperature/$properties → degrees,interval
homie/686f6d6965/temperature/degrees/$settable → false
homie/686f6d6965/temperature/degrees/$unit → C
homie/686f6d6965/temperature/degrees/$datatype → float
homie/686f6d6965/temperature/degrees/$format → -20.0:60
homie/686f6d6965/temperature/degrees → 12.07 
homie/686f6d6965/temperature/interval/$name → "interval"
homie/686f6d6965/temperature/interval/$settable → true
homie/686f6d6965/temperature/interval/$unit →  s
homie/686f6d6965/temperature/interval/$datatype → integer
homie/686f6d6965/temperature/interval/$format → 30:300
homie/686f6d6965/temperature/interval → 60

Keep in mind that the homie-esp8266 implementation does not yet adhere to this version of the Homie-convention.

ingoogni commented 6 years ago

Keep in mind that the homie-esp8266 implementation does not yet adhere to this version of the Homie-convention.

Dat Bert is where I struggled several times now. This morning I did exactly what you wrote above with a bit of "faking it" where "interval/$unit" as a whole becomes a property of temperature instead of "interval" / "$unit"

ingoogni commented 6 years ago

in the super-car example though one would have some trouble? homie/super-car/engine/temperature/$interval → 1s homie/super-car/engine/oil-pressure/$interval → 0.3s homie/super-car/engine/interval/$unit → s

bertmelis commented 6 years ago

You have to respect topology: homie/super-car/engine/temperature/$interval → 1s is actually

homie/super-car/engine/temperature/interval/$name → temp interval
homie/super-car/engine/temperature/interval/$unit → s
homie/super-car/engine/temperature/interval → 1

These settings are bound to the attribute (= interval) and not to the property (= temperature, oil pressure) and certainly not to the node (= engine). So you can have different units and different interval value for each property.

My heating boiler could have the following for example:

homie/heating/outside/temperature/interval → 15m
homie/heating/boiler/temperature/interval → 30s
homie/heating/boiler/waterpressure/interval → 24h
homie/heating/hotwatertank/temperature/interval → 1m
homie/heating/hotwatertank/solartemperature/interval → 30s

All have different units, different values and possibly different ranges. The friendly names are always the same as all the different intervals can be identified by it's parent property.

ingoogni commented 6 years ago

That Bert is clear and obvious from the mqtt point of view. The Homie Convention in that respect though gave me the strong impression that it is limited to one node property without sub properties before the property attribute. Hence my struggle. homie / device ID / node ID / property / property attribute versus homie / device ID / node ID / property [/sub property/...] / property attribute

bertmelis commented 6 years ago

hmmm, now I'm confused myself. 😕

edit: There's no such a thing as a sub property. It's a property attribute. So my previous answers are not correct.

ingoogni commented 6 years ago

Yes, it is a property attribute, regardless of having multiple 'sub'properties or whether they are 'allowed'. This line of thought triggered my initial post. Then I ran into the problem that it is not just a property attribute with a value, but also a property attribute that needs a unit.

So we end up with $interval & $intervalunit and stuff may accumulate into ugliness. Although $datatype and $format are somewhat similar in this regard.

timpur commented 6 years ago

homie/device ID/node ID/property/property attribute: property

My understanding is that it would look like this: We have a device called homie-test we have a node called outsideTemp that node has a property called degrees and interval, thus the structure would look like this.

homie/homie-test/outsideTemp/$type → temperature homie/homie-test/outsideTemp/$properties → degrees,interval homie/homie-test/outsideTemp/$interval → 1.0

homie/homie-test/outsideTemp/degrees/$settable → false homie/homie-test/outsideTemp/degrees/$unit → °C homie/homie-test/outsideTemp/degrees/$datatype → float homie/homie-test/outsideTemp/degrees/$format → -10.0:50.0 homie/homie-test/outsideTemp/degrees → 25.0

Where $interval is part of Node attributes. I dont think $interval should be an property. You set the interval of the node, thus it should be a standard and thus doesn't need to be discoverable. Just like $type. This could be optional, like only enabled for sensor nodes.

eg only if the node in code has enabled a timer task to update the value ( homie-esp8266 3.0 talk ) does $interval become enabled.

Thoughts?

ingoogni commented 6 years ago

Maybe the different view come from how we look at the topics structure. For me sensors or actuators are not interesting, their results are and I name their results after the devices/nodes they are attached to. I'm not interested in the value of brewery/co2-sensors/co2-sensor-1/ I'm interested in brewery/fermenting-room/1-m-high/co2-level and brewery/gas-bottles/1-m-high/co2-level

A practical case, in the brewery there is a plate chiller (device) with in- and out puts for coolant and product (node):

brewery/plate-chiller/coolant-in/temp  -> 12
brewery/plate-chiller/coolant-out/temp -> 72
brewery/plate-chiller/product-in/temp  -> 89
brewery/plate-chiller/product-out/temp -> 18

The most important one:

brewery/plate-chiller/product-out/temp/$unit -> °C
brewery/plate-chiller/product-out/temp/$datatype -> float
brewery/plate-chiller/product-out/temp/$interval -> 1
brewery/plate-chiller/product-out/temp/$intervalunit -> s

The least important one:

brewery/plate-chiller/coolant-out/temp/$unit -> °C
brewery/plate-chiller/coolant-out/temp/$datatype -> float
brewery/plate-chiller/coolant-out/temp/$interval -> 5
brewery/plate-chiller/coolant-out/temp/$intervalunit -> s

This shows that $interval is not a device attribute.

A client listens to the above stream, it has a circular buffer to calculate 5 min averages. Its published output:

brewery/plate-chiller/product-out/temp5avg -> 18
brewery/plate-chiller/product-out/temp5avg/$unit -> °C
brewery/plate-chiller/product-out/temp5avg/$datatype -> float
brewery/plate-chiller/product-out/temp5avg/$interval -> 5
brewery/plate-chiller/product-out/temp5avg/$intervalunit -> min

This shows that the message topology is independent of the actual sensor and can be generated somewhere else and still show properties of a node and its attributes that belong to a device.

That's not all, each in/outlet also has a property flow

brewery/plate-chiller/product-out/flow -> 20,3
brewery/plate-chiller/product-out/flow/$unit -> l/min
brewery/plate-chiller/product-out/flow/$datatype -> float
brewery/plate-chiller/product-out/flow/$interval -> 1
brewery/plate-chiller/product-out/flow/$intervalunit -> s

This shows why $interval is not a node attribute. A node can have several properties that are measured or set out of sync.

Thinking these topics through I work backwards, there is this cold stuff coming out, what is it (property). This cold stuff what do I know about it, temperature, flow -> properties. Measured in what, how, how often -> property attributes. Where does it come from (node) where is it attached to .. etc.

@timpur In your example I would have:

homie/homie-test/outside/temperature
homie/homie-test/outside/temperature/$unit -> °C
homie/homie-test/outside/temperature/$interval -> 60
homie/homie-test/outside/temperature/$intervalunit -> s

Thinking about @fermuch remark, lets put all these data from above in a controller so I can set the product-out temperature. This controller lives in a separate client. It not only 'returns' flow-settings for the inlet valves and pumps but also calculates a quality level. If not good enough it may require to increase the frequency of the temperature measurement/publication.

timpur commented 6 years ago

Intresting, I might think about this one more. You may be right. Either way we go I think this is a good idea, tasks for nodes and update intervals is a nice feature and wanted in homie-esp8266.

see https://github.com/marvinroger/homie-esp8266/issues/257

nicola-lunghi commented 6 years ago

+1 for me

timpur commented 6 years ago

Just need some more opinions, @ThomDietrich @marvinroger ... ?

davidgraeff commented 6 years ago

I think this is very specific and tries to implement a push time configuration attribute. I don't see that most controllers will implement the logic for this, they are mostly interested in discovery via the Homie specification.

I'm closing this for now, but as usual if there is anything to be added to the discussion, please reopen.