search

opendata-stuttgart / meta

Opendata Stuttgart organisiert und reguliert.
125 stars 19 forks source link

Easily download data within location or get server connection #144

Open Nestak2 opened 4 years ago

Nestak2 commented 4 years ago

Hi, I would like to donwnload the all-time data (~200 stations for ~2 years) for the city of Sofia, Bulgaria, and I just don't find an easy way to do this. Is there a way to filter by location or stations ids before downloading? I noticed this issue, but I need to download ~24 such files, each ~13 GB big which is cumbersome. I see that the files contain columns "location", "altitude" and "longitude", so can I filter by them for each day/month in advance before downloading?

It might be easier for me to login into the server hosting the data and download the data I need with terminal commands (I am using Linux 18.04 and know just enough about terminal commands to maybe make this work). Is the connection to the server open and can you give us some instructions on how to connect?

YaguraStation commented 4 years ago

Maybe this can help you as a start: https://github.com/YaguraStation/luftdaten-scripts

Instead of using get-sensor-ids.sh for the whole of Bulgaria, sanitise the list with only Sofia sensors and pipe to get-csv.sh with according time range.

Gladly provide PRs with improvements. The scripts are rudimentary and only covered my specific use case, but a generalised version could benefit the whole project, I think.

EdgarMCR commented 4 years ago

I started writing some helper functions a while back. Its in Python but feel free to copy and use if any is useful: https://github.com/EdgarMCR/luftdaten-plotting-functions

e.g. https://github.com/EdgarMCR/luftdaten-plotting-functions/blob/master/src/get_data.py#L27