[LDI] Reconsider using the data.sensor.community endpoint

earthobservations / luftdatenpumpe

Acquire and process live and historical air quality data without efforts. Filter by station-id, sensor-id and sensor-type, apply reverse geocoding, store into time-series and RDBMS databases, publish to MQTT, output as JSON, or visualize in Grafana. Data sources: Sensor.Community (luftdaten.info), IRCELINE, and OpenAQ.

https://luftdatenpumpe.readthedocs.io/

GNU Affero General Public License v3.0

34 stars 3 forks source link

[LDI] Reconsider using the data.sensor.community endpoint #16

Open amotl opened 4 years ago

amotl commented 4 years ago

We just have been pointed towards an endpoint where data from luftdaten.info is published which we haven't known about yet, e.g. https://data.sensor.community/airrohr/v1/filter/country=BE.

We might consider updating Luftdatenpumpe to use that.

amotl commented 4 years ago

Re-reading https://github.com/opendata-stuttgart/meta/wiki/APIs#accessing-the-api-data says:

There are two domains where the same data is served.

https://api.luftdaten.info - This is optimized for receiving data.
https://data.sensor.community - This is faster and more reliable for serving data, so should be used for reporting cases. Where it is not supported for a certain endpoint, the previous domain must be used.

When comparing LDI API v1 vs. LDI API v2 we see that

https://data.sensor.community/static/v1/data.json - all measurements of the last 5 minutes for all sensors.
https://data.sensor.community/static/v2/data.json - average of all measurements per sensor of the last 5 minutes for all sensors.

So, we may still think about that before switching from v1 to v2, right?

In any case, we might want to switch from api.luftdaten.info to data.sensor.community if this is the designated endpoint for serving data, i.e. it should be used for reporting cases.

cc @wetterfrosch

amotl commented 4 years ago

Also, the filtering endpoint /airrohr/v1/filter listed above might be interesting for different performance improvements, especially when only interested in a subset of data without having to ingest all data.

d-roet commented 4 years ago

perhaps it is not necessary to choose one or the other but to leave the choice to the user by specifying a flag (--LDAPI [v1|v2]) so (s)he can choose between either API?

amotl commented 4 years ago

While api.luftdaten.info is not available today (timeout), data.sensor.community still responds with a valid payload. However, measurements are stale.

cc @wetterfrosch

amotl commented 4 years ago

The documentation makes it more clear now:

There are two domains where the same data is served.

https://api.luftdaten.info This is optimized for receiving data. DON'T use this to request data.

https://data.sensor.community This is faster and more reliable for serving data, so should be used for reporting cases. Where it is not supported for a certain endpoint, the previous domain must be used.