search

opendata-stuttgart / meta

Opendata Stuttgart organisiert und reguliert.
122 stars 19 forks source link

A few questions about the sensors' API and running queries #125

Open aminkov opened 4 years ago

aminkov commented 4 years ago

Hey guys, First of all, congrats on the great air quality tool that you've created! I have a few questions about the API:

  1. How many JSON queries per minute are allowed and does this number depend on the size of the request (i.e. returning the data for 1 sensor loads the server much less than returning data for all sensors)?
  2. Do I have the option to make a query for 1 sensor using the sensor ID, can I make a query for 3, 5, ... a couple of sensors at once by sending their IDs either? Filtering the query by location doesn't work for me, and it doesn't make sense to get the data for all sensors if I need only data for 5 for example.
  3. Any recommendations / good practices when sending JSON requests to your servers?

Thanks in advance and looking forward to your response! BR

ricki-z commented 4 years ago

Returning data for one sensor may not use less resources than downloading the file for all sensors. To return the data for one sensor we use a filter script that needs to go through the whole data file. Downloading this 'static' file (it's generated once per minute) is less work for the server. If you want to download only a part of the data you should consider using the box filter. This filter will use 4 comparisons per date. So with more than 4 sensors it should be faster than downloading those 4 with single requests. Actually the filter at /v1/sensor// will only work for a single ID, but we try to get this working for more IDs at once It should be enough to download the data once every 3-4 minutes. The data.json file and the filter results will contain the data of the last 5 minutes. What do you mean with JSON requests in your last question?

If you want do download the data of 5 sensors it should be okay to use the /v1/sensor/ filter for each sensor ID once every 3-4 minutes. Some users download the data to their home automation software this way for up to 10 sensors.

aminkov commented 4 years ago

Thanks for the quick reply! JOSN requests = HTTP GET requests.

So let's say that I want the data for all 10 sensors in a list. In my app, I would use a loop to iterate the list which will result in generating 10 separate requests (1 for each sensor) in less than a second. So if I do this every 5 mins, if I am getting it right, this won't be an issue for you, right?

10 is probably the maximum number of sensors the app will use and I am really looking for a way to not download all the data for hundreds (in the future probably thousands) of sensors when I need the info just for a bunch of them.

The "box" filter will not work, as I want to filter by IDs, not by location...

ricki-z commented 4 years ago

For a small amount of users this should be okay if you add a small delay between the separate requests. Even a delay of 0.1 seconds should be okay. Its only to spread the requests a bit more.

We try to extend the /v1/sensor/ filter so that you can send a request for more than 1 sensor. So you should look in some months if there is an update on our wiki page about this.