search

Open-EO / openeo-processes-python

A Python representation of (most) openEO processes
Apache License 2.0
11 stars 4 forks source link

Processes sort and order only working for "small extents" #52

Closed ValentinaHutter closed 3 years ago

ValentinaHutter commented 3 years ago

As we are using Dask, the processes that are applied to a dataset are not allow to load any values of the datasets. The only two processes that still load data are the sort and the order processes, so when a large dataset is selected and these processes are applied, to many values will be loaded and the memory usage will be too high. These processes need to be updated without loading any values.

clausmichele commented 3 years ago

Do you have a use case where you use sortand/or order? I've never used them yet.

ValentinaHutter commented 3 years ago

No, there is no use case where we need sort or order yet, I just implemented these two before, but they need to be updated again.

ValentinaHutter commented 3 years ago

Updates of the two processes are now in PR: https://github.com/Open-EO/openeo-processes-python/pull/54

sophieherrmann commented 3 years ago

This was fixed with https://github.com/Open-EO/openeo-processes-python/pull/54