noi-techpark / bdp-commons

GNU Affero General Public License v3.0
2 stars 12 forks source link

meteorology-eurac: Should we combine monthly and daily stations, or just take one of them? #546

Open Piiit opened 2 years ago

Piiit commented 2 years ago

Eurac provides API links for monthly station data and also daily station data. Do I need to sync both of them or just daily station data is enough?

We need to find out if the data is really different, if we need to take both, and or merge them accordingly.

@SaimonasFOS Could you please elaborate on this issue a bit? Which calls are used exactly and what is the difference of response?

Do we have any documentation? website or document?

Piiit commented 2 years ago

@sseppi @rcavaliere FYI

rcavaliere commented 2 years ago

@Piiit we need some data sample to take a decision. Can you get this from the API and share it with us?

Piiit commented 2 years ago

@rcavaliere I do not have any documentation yet, so lets wait for that to proceed...

@sseppi @SaimonasFOS Who could provide some documentation?

sseppi commented 2 years ago

Hi all, here a share the link to the documentation that sent me Eurac https://postgrest.org/en/stable/ Shall we ask them for more infromation/documentation abouth their data?

Piiit commented 2 years ago

OK, the endpoint is this: https://edp-portal.eurac.edu/envdb

So the swagger could be seen like this: https://swagger.opendatahub.bz.it/?url=https://edp-portal.eurac.edu/envdb/

Piiit commented 2 years ago

Example that shows all stations inside climatologies:

https://edp-portal.eurac.edu/envdb/climatologies?select=station

Unfortunately postgrest does not support duplicate elimination or grouping, so we would need to ask to have a special view or function for that inside the Eurac DB or eliminate duplicates on our side

Piiit commented 2 years ago

Some more information:

Piiit commented 2 years ago

WRONG ISSUE, see https://github.com/noi-techpark/bdp-commons/issues/548 instead

First analysis done: I wanted to know if stati..........

Piiit commented 2 years ago

WRONG ISSUE, see https://github.com/noi-techpark/bdp-commons/issues/548 instead

The whole list of stations inside climatologies is this:

Ala: Found
....
SaimonasFOS commented 2 years ago

@Piiit @rcavaliere To elaborate, I was asking this question because I thought that having monthly data could be redundant since we have daily data. You can see the example of this data here: https://edp-portal.eurac.edu/envdb/climatologies But looking at it again, it's not really "monthly" data, but rather climatology (averages from all years in every month). So I assume this is not redundant and could be needed. But if we save this data as measurements, how would timestamps look like? There is no association with any date or time. Some advice is needed from you.

rcavaliere commented 2 years ago

@SaimonasFOS you are perfectly right. We will come up with you with a specification document in which we are going to make a proposal on how the data integration will need to take place