mblackstock / node-red-contrib-influxdb

Node-RED nodes to save and query data from an influxdb time series database
Other
64 stars 44 forks source link

Sanitize input to skip nulls NaN et al #105

Open ouinouin opened 3 years ago

ouinouin commented 3 years ago

Hi, Thanks for this palette i use on a daily basis ans simplifies a lot the logging of PLC while sitting in front of industrial systems :-). I often have issues to do quick flows with the influx palette when the upstream flow sends some nulls in the payload or NaN, those concepts are unknown by Influx, and this leads influx to skip the record, not writing partially the objects that are correct.

Maybe adding some tick box to skip nulls and NaNs , would be a good idea a payload like this : {"L1 Amps":236,"L2 Amps":237,"L3 Amps":237,"Voltage L1":null,"Voltage L2":null,"Voltage L3":null,"Vibration DE":null,"Vibration NDE":null} would become : {"L1 Amps":236,"L2 Amps":237,"L3 Amps":237}

to do this job i actually use this (sorry not a good programmer):

`const asArray = Object.entries(msg.payload); const filtered = asArray.filter(([key, value]) => value !== null); const nonull = Object.fromEntries(filtered);

msg.asarray = asArray msg.filtered = filtered msg.payload = nonull

return msg;`

Easiii commented 2 years ago

A tick box would be very helpful.

However your function doesn't work for me to filter NaN Values.

Do you use this function right befor the InfluxNode?

kr

ouinouin commented 2 years ago

effectively this code sample only treats null and not NaN. i pass all the informations through this function before putting it into the influxdb node. it can also avoids to initilize one db with the wrong "type" on influx since influx seems to autodetect the type of each field based on the first element received (a tick box for converting true/false to 0/1 could also be useful)