Document how or add a function to load NDVI data results from CropSAR process

Open-EO / FuseTS

Time series Fusion toolbox integrated with openEO

https://open-eo.github.io/FuseTS/

Other

22 stars 3 forks source link

Document how or add a function to load NDVI data results from CropSAR process #95

Closed GriffinBabe closed 8 months ago

GriffinBabe commented 1 year ago

When working with the CropSAR process, the results are multiple files that will be downloaded in a folder. Once loaded in memory using for example the xarray.open_mfdataset method, they must be linearly rescaled with two parameters $NDVI{rescaled} = NDVI{original} * 0.00432 - 0.08$

This is not necessarily hard to do, but while the scaling factors can be found in some parts of the repository, they are not explicitly documented in the FuseTS API methods, impacting the user-friendliness of the library.

Alternatively, a function that automatically loads in memory the rescaled NDVI data from a directory path or a list of file paths, could also be useful.

@JanssenBrm

mlubej commented 1 year ago

Additional comments from my side:

I feel like the formula doesn't completely work. after rescaling, I still have some visible shifts with the original NDVI.
in some cases of complex geometries, the downloaded information doesn't contain any mask of the invalid region outside of geometry, and the values there are always 255, there would need to be some mask present to correctly remove the values if someone wants to do spatial aggregation. right now I had to fake the mask by finding pixels where the whole time series is a constant of 255

GriffinBabe commented 1 year ago

It could be also nice to have the output of the process in a single NetCDF format instead of numerous GeoTiff files. There are less files to track that way and I believe that it matches better the xarray.DataArray structure, having all the 4 dimensions (x, y, bands and time) in a single cube

JanssenBrm commented 1 year ago

It could be also nice to have the output of the process in a single NetCDF format instead of numerous GeoTiff files. There are less files to track that way and I believe that it matches better the xarray.DataArray structure, having all the 4 dimensions (x, y, bands and time) in a single cube

I could be wrong here @GriffinBabe , but can't you just specify a netCDF output format when starting the CropSAR batch job as shown here?

JanssenBrm commented 11 months ago

As discussed during the review meeting of the enhancements:

[ ] A new function is added to the FuseTS toolbox that rescales the NDVI values coming from the CropSAR process.
[ ] This function is automatically applied whenever a user executes CropSAR through a local execution.
[ ] The function is added to the CropSAR notebook to showcase how to use it in case of openEO.

JanssenBrm commented 8 months ago

Closing this topic as the internal VITO team is working on a new version of CropSAR where the values aren't rescaled, recognizing that this is inconvenient for users of the service.