search

Open-EO / openeo-geotrellis-extensions

Java/Scala extensions for Geotrellis, for use with OpenEO GeoPySpark backend.
Apache License 2.0
5 stars 3 forks source link

HRL_VPP datacube takes extremely long get RasterSource Extent #250

Closed jdries closed 3 months ago

jdries commented 8 months ago

Collection: EEA_VEGETATION_INDICES

This takes 1 hour per task: https://github.com/Open-EO/openeo-geotrellis-extensions/blob/6d8a11b57e65d42b5941ebf21ba9cf1ba57e0a70/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/layers/FileLayerProvider.scala#L992

Image

Solution could be to have a predefined GridExtent from the catalog: https://github.com/Open-EO/openeo-geotrellis-extensions/blob/6d8a11b57e65d42b5941ebf21ba9cf1ba57e0a70/openeo-geotrellis/src/main/scala/org/openeo/geotrellis/layers/FileLayerProvider.scala#L1125

jdries commented 8 months ago

Data gets loaded in UTM: Created cube for copernicus_r_utm-wgs84_10_m_hrvpp-vi_p_2017-now_v01 with metadata TileLayerMetadata(int16,LayoutDefinition(Extent(572220.0, 5521420.0, 659260.0, 5582860.0),CellSize(10.0,10.0),68x48 tiles,8704x6144 pixels),Extent(636860.6622837363, 5560534.349842346, 658184.4792910906, 5582850.341594372),EPSG:32631,KeyBounds(SpaceTimeKey(50,0,1601769600000),SpaceTimeKey(67,17,1648425600000))) and partitioner Some(SpacePartitioner(KeyBounds(SpaceTimeKey(50,0,1601769600000),SpaceTimeKey(67,17,1648425600000))))

Loading copernicus_r_utm-wgs84_10_m_hrvpp-vi_p_2017-now_v01 with params DataCubeParameters(256, {erosion_kernel_size=3, kernel1_size=17, scl_band_name=SCL, method=mask_scl_dilation, mask2_values=[3, 8, 9, 10, 11], kernel2_size=77, mask1_values=[2, 4, 5, 6, 7]}, FloatingLayoutScheme, ByDay, 6, None, NearestNeighbor, 0.0, 0.0) and bands FAPAR initial layout: LayoutDefinition(Extent(572220.0, 5521420.0, 659260.0, 5582860.0),CellSize(10.0,10.0),34x24 tiles,8704x6144 pixels)

jdries commented 8 months ago

It turns out that this slow code path was reached because the fapar load_collection had 'mask_scl_dilation' settings, which won't work, but also causes the requirement to load the gridExtent from geotiff metadata, which is very slow here.

After removing mask_scl_dilation, the step that took 1.9 hours finishes in a few seconds, and also the actual data loading continues with 'normal' timings, albeit a bit slower than what we typically see for Sentinel-2 reading.

Still very unclear why reading geotiff metadata from object storage would take an hour and then still manage to finish...

JeroenVerstraelen commented 4 months ago

Grassland watch wants to use this so we will want to solve this asap. We need to log an issue with Cloudferro.

Maybe read geotiffs with GDAL geotiff uses geotrellis custom reader JPEG2000 + netcdf: GDAL

jdries commented 4 months ago

I ran a test over a larger area on: COPERNICUS_R_UTM-WGS84_10_M_HRVPP-VPP_P_2017-NOW_V01 Performance was fine, even very good. Now perhaps also need to check ppi trajectory.

jdries commented 3 months ago

we now also got a report for this job: j-240430ad01e14ca5a6c8ba6727d669cf where reading hrl vpp ppi took 1 hour per task

2 days later the same ppi was much faster. I did spot that we are still using UrlConnection based S3 reading, now switching to httpclient.

jdries commented 3 months ago

Latest job on staging finished drastically faster: j-2405037dccca4849af7c22456b1b6984 Not sure if this is performance fluctuation or the change of http client.

jdries commented 3 months ago

Ok: I ran a job on prod and on staging at the same time. The one on prod is still very slow, the one on staging is faster.

jdries commented 3 months ago

closing again, needs reopen if an issue is detected again