psi-4ward / AskSinAnalyzerXS

Analyzer for radio telegrams in a HomeMatic environment
Other
48 stars 7 forks source link

Write data to InfluxDB2 + Grafana Dashboard #98

Open bolausson opened 1 year ago

bolausson commented 1 year ago

Hello all,

just in case someone would like to track the telegrams via InfluxDB2/Grafana.

I wrote a little Python script which can parse the CSV files and write the data to InfluxDB2 if "Persistent data storage" is activated in AskSin Analyzer XS. "Payload" and "RAW" data is not sent to InfluxDB - but can easily be enabled by removing the comment from two lines.

AskSinAnalyzer-CSV-to-InfluxDB

Grafana Dashboard

Greetings, Bjoern

psi-4ward commented 1 year ago

Nice 😉

bolausson commented 1 year ago

I tried to implement it within the persistentStorage.ts but since I never coded something in TypeScript I scraped the effort and took the detour via the CSV files + Python which just took a couple of minutes.

I mean there is a InfluxDB2 client available so I might give it another try when I have more time 😄

psi-4ward commented 1 year ago

Describe the protocol to me and the mapping you've choosen. Probably I will implement it. Tbh, I stored the data into influx for some timing using the web socket and node red.

bolausson commented 1 year ago

The mapping is a simple dictionary which replicates your CSV header one to one. Payload and RAW should probably be optional to not bloat your TSDB.

MM = [
    {
        "measurement": 'Telegrams',
        "tags": {
            "flags": str(row["flags"]),
            "type": str(row["type"]),
            "fromAddr": str(row["fromAddr"]),
            "toAddr": str(row["toAddr"]),
            "fromName": str(row["fromName"]),
            "toName": str(row["toName"]),
            "fromSerial": str(row["fromSerial"]),
            "toSerial": str(row["toSerial"]),
            "toIsIp": str(row["toIsIp"]),
            "fromIsIp": str(row["fromIsIp"]),
#          "payload": str(row["payload"]),
#          "raw": str(row["raw"]),
            },
        "fields": {
            "tstamp": int(row["tstamp"]),
            "date": str(row["date"]),
            "rssi": int(row["rssi"]),
            "len": int(row["len"]),
            "cnt": int(row["cnt"]),
            "dc": float(row["dc"]),
            },
        "time": int(row["tstamp"]),
    },
]

Not sure if the translation below to JS is correct:

import {InfluxDB, Point, HttpError} from '@influxdata/influxdb-client'
import {url, token, org, bucket} from './env.mjs'

// If you use the timestamp like in your CSV file, you must tell InfluxDB to use write precision='ms' else the write fails
const writeApi = influxDB.getWriteApi(org, bucket, 'ms', writeOptions)

const point1 = new Point('Telegrams')
// Add all tags:
  .tag('flags', flags.toString())
  .tag('ftype', type.toString())
  .tag('fromAddr', fromAddr.toString())
  .tag('toAddr', toAddr.toString())
  .tag('fromName', fromName.toString())
  .tag('toName', toName.toString())
  .tag('fromSerial', fromSerial.toString())
  .tag('toSerial', toSerial.toString())
  .tag('toIsIp', toIsIp.toString())

/*
  not sure how to implement a clause to add the paylod and raw data if a checkpox for one or the other is checked
  .tag('payload', payload.toString()) 
  .tag('raw', raw.toString())
*/

// Add all fields
  .intField('tstamp', tstamp)
  .stringField('date', date)
  .intField('rssi', rssi)
  .intField('len', len)
  .intField('cnt', cnt)
  .floatField('dc', dc)

// Add the timestamp of the telegram
 .timestamp(tstamp)

// write the data
writeApi.writePoint(point1)

There are lots of examples how to use the Node.js client: https://github.com/influxdata/influxdb-client-js#usage Write data with the InfluxDB JavaScript client library and the API Reference