search

urbanobservatory / standards

Standards and schema documentation for the observatories programme
2 stars 0 forks source link

Observatory data federation vs. central repository #25

Open Joe-Heffer-Shef opened 4 years ago

Joe-Heffer-Shef commented 4 years ago

Hi,

I noticed in the draft document that it is assumed that the data generated by each observatory will stay located there i.e. it won't be copied to a central repository first. Is this an architectural decision that has already been made, or just a working assumption?

It says in the architecture section:

A federated approach is adopted.

lukeshope commented 4 years ago

Hi Joe,

It's a decision that's been made. This reflects the heterogeneity across the observatories, both with respect to the sensors they have and the types of data they generate (i.e. some are numeric timeseries, while some are raster files from radar scans etc). It is expected that each observatory may have more than one API in operation, as part of a service-oriented architecture. They may even have a different API for each type of sensor.

Logistically, there is no shared IT facility across the observatories, and would be no funding to use cloud services beyond FY2020-21, so it would make centralising the data very difficult in any case. There are discussions with DAFNI about whether they will take bulk files from the UOs into their system, but they haven't expressed any desire to host our APIs thus far.