eurodatacube / eodash

Software behind the RACE dashboard by ESA and the European Commission (https://race.esa.int), the Green Transition Information Factory - GTIF (https://gtif.esa.int), as well as the Earth Observing Dashboard by NASA, ESA, and JAXA (https://eodashboard.org)
https://race.esa.int
MIT License
90 stars 43 forks source link

Define possible programmatic access to data concepts (GTIF API) #2269

Open santilland opened 1 year ago

santilland commented 1 year ago

Ticket for requirements collection for data access

Patrick1G commented 1 year ago

@slumnitz @santilland

Access to GTIF data and functionality was one of the most prominent requirements brought forward during the user consultation. We should collect the specific requirements from different users how they want to programmatically interact with GTIF.

For now, from my point of view, and all TBD:

  1. provide an abstraction layer on top of existing APIs such as SH or GeoDB, so that users might be able to formulate requests with a simple client library or command line interface
  2. provide access to key capabilities e.g. the site suitability assessment for renewables via the library
  3. provide access to GTIF datasets in a way that makes it hard to extract/download the entire data population
  4. prospectively, assuming that some capabilities will be transformed into on demand services (e.g. ML based energy infrastructure mapping), also provide programmatic access to those on demand services.
santilland commented 6 months ago

Hello @patrick-griffiths touching on this again since it came up per email, it is possible to access the GeoDB data through a RESTful API, in principle as well as SH data. So this is possible for example for the Green roof and PV detections datasets. We can create some information on how to access the data with some query examples for regions. There is no data limitation concepts, but it will probably happen that too large requests timeout and are not serviced. The problem is knowing how large a query result actually is before actually running the query. We could consider creating a very tailored endpoint that only accepts some very specific parameters, such as NUTS id and if green roof or PV data is wanted, and that would respond with a geojson for the selected region. For really making sure we limit data access to a specific "size" we would need to introduce user management to keep track of who accessed what. So i think these are the 3 levels we have:

Patrick1G commented 5 months ago

some more requirements from potential users:

In my research, I came across the GitHub repository at https://github.com/dcs4cop/xcube-geodb/tree/main , which seems to be the one you're using.Would it be alright for us to utilize this WMS for accessing the data?

To align with our current data integration processes, we would ideally access the GTIF data via Web Map Service (WMS), Web Feature Service (WFS), or in a GeoJSON format. Could you provide URLs or endpoints for these services, if available? This would greatly facilitate our integration efforts.

Regarding our tool, we are developing a web application aimed at aggregating all available data in Austria for renewable energy. Currently, our primary data source is https://data.gv.at/ and https://www.openstreetmap.org/ but are also using other open web service from https://www.geologie.ac.at/services/web-services and many more.

Patrick1G commented 4 months ago

Another user would like access to GTIF data via WMTS:

Dear GTIF Team,

I am writing on behalf of Schedlmayer Raumplanung ZT GmbH, a company specializing in spatial planning and consultancy based in Austria.

We have recently come across the Heat Explorer tool available for Austria and found it immensely useful for some of our studies. Given the nature of our projects, integrating this tool with our own GIS system would significantly enhance our ability to analyze and visualize data effectively.

We are interested in learning if there is a possibility to access the Heat Explorer tool via a WMTS server or any similar service. Such access would allow us to seamlessly combine the tool's capabilities with our datasets in our GIS environment, thereby enriching our analytical processes and outputs.

Could you please provide information on whether this type of access is available or if there are any alternative methods to integrate the Heat Explorer tool with external GIS systems?

Thank you very much for your assistance and the valuable work you are doing. We look forward to the possibility of leveraging the Heat Explorer tool to its full potential in our projects.

Best regards,

santilland commented 4 months ago

Hello @Patrick1G , the heatmap data is provided directly by Geoville, it was integrated into the GTIF client, but it is not served by the GTIF infrastructure. As far as i understand, this dataset can't be integrated somewhere else unless allowed by Geoville