search

opendata-stuttgart / sensors-software

sourcecode for reading sensor data
573 stars 312 forks source link

Work in progress: support for InfluxDB and MQTT as backend, new WiFiConfig #33

Closed ricki-z closed 4 years ago

ricki-z commented 7 years ago

These are some enhancements that will come with the next versions:

ensonic commented 7 years ago

What do you mean by "too many different payload strings and auth methods"? If this gets dropped, whats the suggested way to log the data to some place where I can get history and e.g. graphs?

ricki-z commented 7 years ago

Some mqtt brokers need user name and password, others need a device id and a device password. I've requests for a payload with a special format and a payload with json formatted data. You can easily implement a "proxy", that translate json to mqtt. Then you can use this proxy as custom api. Historic data can be found here: http(s)://archive.luftdaten.info/ Graphs can be found here: http(s)://www.madavi.de/sensor/graph.php "Life" data can be found here: http(s)://api.luftdaten.info/static/v1/data.json . This file is updated every minute and contains all sensors and values sent to the server in the last 5 minutes.

ensonic commented 7 years ago

Thanks for the explanation, makes sense. I'll look into the custom-api suggestion. Is there an irc / gitter.im chat for this project (so that I don't hijack more threads :))?

ricki-z commented 7 years ago

We use slack. Please send a short mail to our info address with the request for an invitation.

amotl commented 7 years ago

Dear Rajko and the whole team of luftdaten.info,

first things first: Thanks for conceiving this great project!

Together with @einsiedlerkrebs, we recently have been able to dedicate some time to consuming and processing the data you are publishing.

The outcome of that work is called Luftdatenpumpe. It can read measurement data from the luftdaten.info live api and republish it to MQTT, essentially implementing the HTTP to MQTT convergence like outlined in your post above.

Cheers, Andreas.

P.S.: Along the lines, Luftdatenpumpe can also write the data to an InfluxDB database and is able to provision an appropriate Grafana dashboard to display historical data from all sensors by geographical location.

Maybe this can spark your interest, it could become it a meaningful supplement to the luftdaten.info map you already provide with more flexible access capabilities compared to the current RRDTOOL-based graphing.

To get more into the details, you might want to have a look at about luftdaten.info in InfluxDB and Grafana. We are happy to receive any feedback and answer further questions about this.

njumaen commented 7 years ago

@amotl This ist soooooo awesome!!! Great work!

tho-wa commented 7 years ago

Is it down actually @amotl ?

amotl commented 7 years ago

Dear Thomas (@tho-wa),

you are right, very sorry. We expanded the disk size of the machine before the weekend and missed to spin up the (separate) Grafana instance again. Now its up and running again, thanks for noticing!

Cheers, Andreas.

rzr commented 5 years ago

Basic MQTT support is still welcome, at least as well as WebThing REST API

Some hints at:

ricki-z commented 5 years ago

@rzr please read the complete thread. There are good reasons why we don't support MQTT (memory usage, different configs for different platforms, and if we start with one platform how should we argue against others)

rzr commented 5 years ago

Well i assumed there were traction for MQTT support:

So I guess it should be implemented in per platform custom fw,

I am targeting this device: ESP8266 ESP8266EX LoLin NodeMcu v3

Any clue ?

ricki-z commented 5 years ago

You could modify the "Own API" functionality. Or set up a mapper JSON to MQTT and use this as an "Own API" endpoint. There is more than just implementing MQTT. This protocol is very unspecific about the data format. And: The source code should work on all ESP8266 systems with more than 1MB flash memory. There is no need for a per platform fw.

dirkmueller commented 4 years ago

Looks like nothing is still open here mqtt had been removed (although it could be added back now that we have fewer constraints on memory)