geotiff + netcdf: support writing sub-day files

Open-EO / openeo-geotrellis-extensions

Java/Scala extensions for Geotrellis, for use with OpenEO GeoPySpark backend.

Apache License 2.0

5 stars 3 forks source link

geotiff + netcdf: support writing sub-day files #257

Closed jdries closed 3 months ago

jdries commented 6 months ago

I propose to detect if timestamps have hours and or minutes, and in that case use a more accurate timestamp or resolution.

Here: https://github.com/Open-EO/openeo-geotrellis-extensions/blob/d618910335cb0e5e7d6e00d5903091d410802015/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/geotiff/package.scala#L127

and here:

https://github.com/Open-EO/openeo-geotrellis-extensions/blob/01b67c85f27ff59fde22872a53a13306b79eb6f9/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/netcdf/NetCDFRDDWriter.scala#L130

EmileSonneveld commented 6 months ago

I removed the aggregate from https://github.com/Open-EO/openeo-geopyspark-driver/issues/647 to test:

When exporting as tiff, I indeed got only a date in the name, not hour and minutes.
When exporting as NetCDF, I got a combination of both samples in the same file:

EmileSonneveld commented 5 months ago

There still seems to be a problem with nodata in .nc files

EmileSonneveld commented 5 months ago

This was implemented for NetCDF, but still need to find a way to implement it for TIFF without performance penalty

jdries commented 3 months ago

I pushed the basic solution for writing geotiff per date, ignoring that different date formats will be used if an rdd mixes day and subday timestamps.

jdries commented 3 months ago

it works on staging for gtiff, there's also a unit test, so closing this one now