earthpulse / eotdl

Earth Observation Training Datasets
https://eotdl.com
MIT License
17 stars 6 forks source link

feature engineering via openEO UDPs #142

Open Patrick1G opened 6 months ago

Patrick1G commented 6 months ago

At the SRR we discussed how openEO (and SH statistical API) could support feature engineering functionality in EOTDL.

Here is an example of using standard openEO functionality, which created a process graph: https://docs.openeo.cloud/usecases/crop-classification/#computing-temporal-features

Here is a more advanced example that uses feature engineering via openEO User Defined Processes (UDP): https://github.com/Open-EO/openeo-community-examples/blob/main/python/Sentinel1_Stats/Sentinel1_Stats.ipynb

In addition, we have developed some data fusion capabilities as openEO UDP, which perform data fusion SAR-optical and gap filling: https://github.com/Open-EO/FuseTS/tree/main/notebooks/OpenEO

juansensio commented 6 months ago

The implementation of the feature engineering functionality will be divided in the following structure:

  1. Data fetching from Sentinel Hub with openEO a. Sentinel 2: bands, level, resolution, clouds b. Sentinel 1: filtering by acc/desc nodes, polarization, product type
  2. STAC extension for feature recipes (both training and inference) a. Data transformations (vegetation indices, texture, …) b. Temporal metrics (percentiles, min/maz stats over time) c. Investigate reponse time for incerence
  3. STAC extension for Feature stores
  4. A dedicated session with Sinergise/VITO will be scheduled to work on this topic. It is important to remark that EOTDL is not responsible for developing feature engineering functionality, but to interface with the existing ecosystem based on openEO. The development of feature engineering shall be guided by user stories and actual use cases.

Patrick1G commented 5 months ago

this looks like a good plan! copy: @jamesemwheeler @dmoglioni