search

zensqlmonitor / influxdb-zabbix

Gather data from Zabbix back-end and load to InfluxDB in near real-time for enhanced performance and easier usage with Grafana.
MIT License
72 stars 26 forks source link

Some values does not reach InfluxDB #13

Open vasekch opened 7 years ago

vasekch commented 7 years ago

Hi, I have observed this strange behaviour.

Some values does not reach InfluxDB and remains only in Zabbix. See screenshot from Grafana comparing two different sources of the same metric.

image

I use setup with Zabbix proxies and only hypothesis I can come up with is that my metrics are slightly delayed and sometimes data arrive with older timestamp than current Startdate of influxdb-zabbix.

Also suspicious is the pattern how missing data occur. My thinking here is that my metric is collected every 30 sec and influxdb-zabbix poll interval is set to 15 sec so this interfere together somehow.

I'm thinking that better approach for me may be to use database max(ID) field instead of timestamp for last synced record pointer as some of my data may arrive on time and some with considerable delays. Records that arrive on time will raise Startdate timestamp too high and prevent older measurements arriving later from processing for me.

Thanks for opinions.

vasekch commented 7 years ago

I have forked the repo and redesigned this to use auto-increment columns instead of timestamps. Updating current version to provide both fucntionality had too big overhead as I'm not a real go programmer. Mine version is to be found on https://github.com/vasekch/influxdb-zabbix Thank you @zensqlmonitor for the kick-start! Any comments welcome! (expecially regarding how inefficient my auto-increment colums are)