Use of Tags

[Sineos]

I'm processing some values in Node-Red and return the result to ioBroker. Now I would like to write this new data point InfluxDB but I would need to be able to use tags alongside of the value as specified in Influx Line Protocol. Something like:

{
  tag1: 'bla'
  tag2: 'blubb'
  value: 15
}

Is this possible?

[Apollon77]

currently not. This would be a feature request where the main question would be how to do it data-wise ... so where the tags come from when normal values are updated? So a generic concept for this would be needed.

Basically it could be possible with javascript adapter dirctly ynd a influxdb npm package ...

[Sineos]

For now I have solved my case within Node-Red, nevertheless I would very much appreciate a twofold approach:

1. Value as array / object If the ioBroker value is an array, handle the case analogous to the node red influxdb node, e.g.

[{
    value: 12,
},
{
    tag1:"sensor1",
    tag2:"device2"
}]

2. Allow attributes to be passed as tags Allow to define attributes in the settings that are then passed to influxdb as tags, e.g.

This would allow to efficiently use the influxdb tag system, e.g. multiple temperatures in a heating system could be written to one measurement, distinguished by tags.

[Apollon77]

Hm ..

Do I understand you correctly that you would like to have multiple datapoints with the same AliasId and then use different tags too differentiate the logged data? is this your goal usecase?

To your ideas: Case 1 using array/objects as state values are really special cases not used by adapters or such, and if only for "Internal" stuff. SO this then would only be useable for own states via Javascript or such and also then no Visualisation UI can do anything with these states, so they would then only be for logging ... this is a very limited usecase

Case 2 is kind of the same. "common" is the wrong place for the data fields because common are the "iobroker defined meta fields" and your custom ones are not defined, so may not be persisted. "native" is owned by the adapter developer and if add something there the developer do not expect noone will ensure that the data stay persisted. In both cases they could sometime interfere with iobroker- or adapter owned fields with the same name, so generally bad idea 8when looking into future)

So the only way would be to put this completely into common.custom.influxdb.X ... and this would mean to enhance the "edit influxdb settings for this datapoint" UI to allow adding key/value pairs for additional static tags, so a bit like you sggested there, but as object key-value instead of an array.

If you like to try changing it I would accept a PR ... Alternative I could put it on my todo list and will check back when I find time

[Sineos]

Do I understand you correctly that you would like to have multiple datapoints with the same AliasId and then use different tags too differentiate the logged data? is this your goal usecase?

Exactly. InfluxDB is more efficient if the data is stored in a single series than in different measurements. Also potential calculations (sum, median, math functions) is more easily done (or supported at all) within one series.

SO this then would only be useable for own states via Javascript or such and also then no Visualisation UI can do anything with these states, so they would then only be for logging ... this is a very limited usecase

Agreed. Only for special cases anyway and can be done quite good in node red.

Case 2 is kind of the same. "common" is the wrong place for the data fields because common are the "iobroker defined meta fields" and your custom ones are not defined, so may not be persisted. "native" is owned by the adapter developer and if add something there the developer do not expect noone will ensure that the data stay persisted. In both cases they could sometime interfere with iobroker- or adapter owned fields with the same name, so generally bad idea 8when looking into future)

Maybe a misconception on my side: Does this function not allow to add arbitrary attributes to a state? If I add attributes here, they will end up in the common element.

So the only way would be to put this completely into common.custom.influxdb.X ... and this would mean to enhance the "edit influxdb settings for this datapoint" UI to allow adding key/value pairs for additional static tags, so a bit like you sggested there, but as object key-value instead of an array.

Something like:

"custom": {
      "influxdb.0": {
        "enabled": true,
        "changesOnly": true,
        "debounce": "1000",
        "retention": 0,
        "changesRelogInterval": 0,
        "changesMinDelta": 0,
        "storageType": "",
        "aliasId": "WM_Luftdruck",
        "tags": {
            "tag1": value1,
            "tag2": value2
        }
      }
    }
  }

Would be fine as well, thou not as elegant as when attributes could be used.

If you like to try changing it I would accept a PR ... Alternative I could put it on my todo list and will check back when I find time

Well, I wish I had the skills in js and experience in ioBroker to quickly hack this away, but my first scanning of the code ended with "um, yeah, well...looks like js" So I would accept to end up on your surely significant todo list :-)

[Apollon77]

Does this function not allow to add arbitrary attributes to a state? If I add attributes here, they will end up in the common element.

In fact yes it allows that, but also is a generic functionality. As said when the object is "owned" by an adapter then it may not be the best idea to do such free editing because no one will tell you what the behaviour will be for this certain adapter! That's why I would mainly use this for "own" objects.

Would be fine as well, thou not as elegant as when attributes could be used.

In my eyes this the best solution because this tagging is only related to the certain influxdb logging, so there is no real benefit in having these infos in a different place.

[nisiode]

Hi,

any news when this will be implemented? I can help in testing+development...

Just switched hole stats to influxdb+grafana. Usage of tags would ease the entire reporting.

Why not build a series for each common.role and put the ID as tag? Maybe a tag for room and function would help aswell.

[Apollon77]

Answer: as soon as I have time for it. Have some higher prios. I also will accept PRs if you want to try it

[nisiode]

Started to add tags in a fork: https://github.com/nisiode/ioBroker.influxdb

What's changed:

• Removed AliasId
• Added instead of aliasId the measurement which is by default set to common.role
• Added four tags:
  1. adapter-name
  2. instance-number
  3. state-id
  4. custom name (by default set to common.name)

Yet not covered:

• everything around getHistory()

[Apollon77]

Hey, thank you! AliasId should not be removed! Please re-add. This feature is needed also for oTher topics like device replacement and is an existing feature I do not want to remove. Also because it is a consitent feature in all 3 history adapters. This should stay "in sync".

Rest sounds like to stay backward compatible for now :-) Or?

What changes to getHistory you had in mind?

[nisiode]

The feature "AliasId" still exists, but this now is called "measurement". Measurement is an existing keyword for that part of a point at influxdb. => https://docs.influxdata.com/influxdb/v1.7/concepts/key_concepts/#measurement

Agreed, everything should be backwards compatible.

Nothing planned to change in getHistory, but so far haven't had a look at this part.

[Sineos]

Many thanks for working on this one. I would love to have at least two custom tags. I like to organize my influxdb data in the following way:

Measurement: roomClimate

{
  tag1: 'humidity',
  tag2: 'office',
  value: 45
},
{
  tag1: 'temperature',
  tag2: 'office',
  value: 22
}

[nisiode]

Why not use humidity and temperature as measurements? Would suggest not to mix different units in one measurement.

Office could then be set as the tag "name" for both measurements.

[Sineos]

Would be an option of course. A while ago I did some performance and size measurements and found influxdb more efficient in both terms when using the same measurement and distinguish with tags. I guess this is due to the fact that tags are always static and measurement variable.

[Apollon77]

Great work for such a complex Adapter :-))

Some comments: For Alias: In favor of comparism for the user between the three history adapters it would be great to have "Alias ID" somewhere at least in Admin-Config UI in the displayed text or so :-) And for backward compatibility please reuse the object key name because else all users that configured aliasIds will loos all of them. This should not happen! If it is really the same equivalent feature then it should stay as it was before from a user perspective. You can change the internal name, but then custom*.html need to "convert" it, but also in the adapter itself you need to do that.

And yes when you do it that way you need to add a comparable handling for getHistory so that it acts as the alias Feature was - which mainly means that a getHistory for the state ID in reality uses the measurement which was set if it is different

Why you have remove the tags q, ack and from? Please re-add them!

I also think that adding reall custom key/vakue to the adapter config should not be that high effort ... then it is really flexible ... or ?!

Text translations can be done by http://iobroker.net:3000/

When you enable Travis-Ci for your fork you can execute the testing before doing the PR ... else you are welcometo make a PR and use the officialtravis to iterate till this works too :-) You may need to change the testing because of the field rename ... or better enhance the testing to test that aliasId and mesaurement as names exist.

[raintonr]

I notice these comments stopped over a year ago. Do you need help with this?

I can make my InfluxDB/Grafana system work as required right now easily enough (simply wrote a custom script to write send objects to InfluxDB with the tags I require), but... if this module supported tags then we could give that functionality to everyone.

Happy to fork the repo and make a pull request if you're open to this.

[Apollon77]

It is just a matter of time and priorities :-(

But yes PR would be awesome. I think the challenge is to stay backward compatible... or?!

[raintonr]

No worries. So what I think should happen is:

Alias - you want to keep this for compatibility, etc. Cool, but this is actually the measurement name being used in InfluxDB. If everything else is left, at least in the UI this should read, "Alias (InfluxDB Measurement)"?

Custom Tags - IMHO, we simply need to add another per-object configuration item that is a freeform string into which users can type the tags they require. Then make these flow directly to InfluxDB.

Eg. One could have 2 objects, both with an Alias of 'temperature' (ie. these should be written into the InfluxDB 'temperature' measurement). The 'Custom Tags' for each could be:

• room=kitchen
• room=bedroom

But if one had more complicated devices (we have a heat pump boiler for example, which has a number of temperature sensors), the objects could still have the 'temperature' alias but with more tags such as:

• device=PAC,sensor=LWT
• device=PAC,sensor=Inlet

etc, etc.

If you agree with this approach happy to make a PR.

P.S. Oh, I also have a slightly convoluted requirement to 'timeshift' a handful of our values (ie. add a number of hours to the timestamp as they are written). I would also then add a 'timeshift' option on each object which would be a number of seconds to adjust the timestamp by when writing (default to zero of course).

[nisiode]

@Apollon77 not sure if the AliasID in each adapter is still needed since iobroker has implemented this as a core feature now?

Currently I would suggest the following mapping: iobroker => influx measurement: common.role => measurement tags: adapter name => tag "adapter" instance number => tag "instance" _id => tag "id" common.name => tag "name" free text => tag "custom" values: val => value "val" ack => value "ack" q => value "q" from => value "from"

Maybe the device name could be added as tag for gruping in charts. @raintonr What exactly would you suggest as a mapping from iobroker to influx?

[raintonr]

Well, I'm using ioBroker for home automation so have a number of values recorded. Some examples are:

1. Temperatures in various rooms and sensors on our boiler, etc.
2. Humidity.
3. Electrical power on various circuits, but also power production by our heat pump in various modes (heating, cooling, hot water production).
4. Water flow (in the boiler) or air flow (in our ventilation).

I believe InfluxDB recommend grouping measurements of the same type. So in the above one should have 'temperature' as a measurement and use tags to define the other attributes. It doesn't matter if the temperature measurement is the air temperature in a room, or water tank temperature in the boiler, etc. They are all temperatures.

To separate further using tags for 1 & 2, obviously one would like to record the room being monitored (side note, I did think it might be nice to have an option to add the existing room & function that apply to all objects as tags). If the temperature is in a device (our heat pump or ventilation system) then that tag ('device') is also used.

For 3 & 4 I use 'power' or 'flow' as the measurement and then 'circuit' works as a tag for both electrical and air or water circuits.

Apologies if I'm not being very clear. I will just share this Gist which is the script I use to map from ioBroker objects to Influx measurements and send them, hopefully you get the idea...

https://gist.github.com/raintonr/e319c2e841a1969a61ed7bd784fd8388

[raintonr]

@Apollon77 Do you have any opinion on what I said above about the proposed solution? I am willing to code this and make a pull request but would prefer if one of the repository owners confirms they are happy with this approach before I do this, because it would be a waste to make such changes for them to be rejected.

Thanks :)

[pedrr]

I really would love the feature to add more tags (beside, acc, from and q) to the measurement.

I never liked the concept of aliasID in the Database. This would mean, that it is up to me to ensure that there are no duplicates in my complete ioBroker instance, cause aliasID overrides the objectID

My UseCase: I like to stay with the objectID as identifier, but be able to add a more descriptive name to id Currently my timeseries in Grafana shows "modbus.1.holdingRegisters.44002_SG-READY-EINGANG1" or "plenticore.0.devices.local.battery.Charge_P" in the Legend. I'd love to see "SG-Ready1" or "Battery Charge" instead. From my understand this could be the

[Apollon77]

in the Legend.

You know that fancy Grafana feature that you can define the name to show in legends - per chart?

[Apollon77]

So you would like to get a Tag with the state name or what? But that name has nothing to do with the timeserias data because it belongs to the "datapoint". SOrry I do not get that particular request. Tags for "time serial relevant data" ... ok ... but for "meta data of the datapoint"???

[Apollon77]

In fact regarding this issue at all: In the latest version AND with InfluxDB 2 tags are supported ... just not "custom" ones

[pedrr]

in the Legend.

You know that fancy Grafana feature that you can define the name to show in legends - per chart?

Yes, but in fact then I have to manage those data in two sources (ioBroker and Grafana). I'd like define the whole dataset in one source, so ioBroker should provide the datapoints and a discriptive name for the datapoint (as tag)

I'd like to have something similiar to AliasID, but implemented as Tag. (benefit: ioBroker manage to have the ID unique)

how do I use tags with the latest version (sorry, for the stupid question here ;))

[Apollon77]

how do I use tags with the latest version (sorry, for the stupid question here ;))

In adapter settings in enhanced settings you can enable it for an empty/new database

[pedrr]

ah, yes I did so. but how can i add Tags to my datapoints?

[Apollon77]

As said in my post: Adding custom tags is still not implemented ... but the other data are stored as tags "already", so it could be easier now to add such a feature

[swissglider]

it would also be very interesting if we can have the corresponding enums also send as tags, i.e.;

• rooms
• functions

And also meta data like:

• adapter name
• device name
• channel name
• device name

In this way you can do cool filters on grafana or so.

Till now I do this while I have cryptic alias names with all the information on it like and handle it in grafana:

i##name:Hue Bridge##stateName:Bewegungsmelder Gang Mitte.presence##channelName:Bewegungsmelder Gang Mitte##deviceName:Hue Bridge##adapterName:hue.0##id:hue.0.Bewegungsmelder_Gang_Mitte.presence##type:state##function:enum.functions.motion

[Apollon77]

adapter name (or at least an identifier for it) should be "from" and existing already. device/channel/state names would only work for adapters that use such a structure and is near to impossible to know/guess

[swissglider]

adapter name (or at least an identifier for it) should be "from" and existing already. device/channel/state names would only work for adapters that use such a structure and is near to impossible to know/guess

Many thanks for your answer, is it possible that we can have i.e a second field like alias name but not as _measurement but as a tag ?

[Apollon77]

Can you please check the possibilities of 2.x of the adapter and tell whats pot missing?

[rolouis]

Are there any updates on this issue?

[jb-io]

Are there any updates on this issue?

Yes. I created a pull request https://github.com/ioBroker/ioBroker.influxdb/pull/278 with a working implementation. The only (known) thing not working is the automated testing.

If you like to, you can have a look into the pull request, try it out and give feedback if there are any failures.

[Apollon77]

As soon as I find some time again I will also look or maybe just accept the PR ... buet there also is an other topic "to be done" where I need time ... Sorry ... be patient please pr try the PR/Fork from @jb-tool