pydata / xarray

N-D labeled arrays and datasets in Python
https://xarray.dev
Apache License 2.0
3.63k stars 1.09k forks source link

Sliding window (mix of rolling and coarsen) #9696

Closed JulienBrn closed 3 weeks ago

JulienBrn commented 3 weeks ago

Is your feature request related to a problem?

I often have functions to apply to sliding windows, such as a fft computation, but both coarsen and rolling do not fit. For example, generating a spectrogram with overlap between windows is complicated using xarray. Of course, one could use scipy STFT if we consider the example of the spectrogram, but one still needs to manage the coordinates manually.

Describe the solution you'd like

A new window function that combines rolling and coarsen. The new window would have the current dim parameter of rolling and coarsen split into two:

This unifies rolling and coarsen as rolling is simply hop=1 and coarsen is hop=window_size.

As for the implementation of this datastructure, I suspect we could use as_strided for very efficient implementation of construct.

Describe alternatives you've considered

Additional context

No response

max-sixty commented 3 weeks ago

[edit: posted before @dcherian 's response below, his response is more helpful...]

To the extent this is a popular request across lots of fields, we could have it in xarray.

But assuming it's niche-r (I'm less familiar with this routine, as one small datapoint), I'd suggest starting it as a function outside xarray. That doesn't require so much consensus and the API can evolve more easily. If it becomes widely-used or load-bearing, we could then promote to xarray itself.

https://xarray-extras.readthedocs.io/en/latest/ is one place, or could even just start as a gist.

(This is one view among many, I don't think we have an official policy on what to put in vs outside xarray, keen to hear others' thoughts...)

dcherian commented 3 weeks ago

The stride kwarg to construct does what you're asking for IIUC: https://docs.xarray.dev/en/stable/generated/xarray.core.rolling.DataArrayRolling.construct.html . We already use sliding_window_view under the hood.

This tutorial may be useful: https://tutorial.xarray.dev/intermediate/01-high-level-computation-patterns.html#index-space-windows-of-fixed-width

headtr1ck commented 3 weeks ago

There is this old issue: https://github.com/pydata/xarray/issues/3608 Maybe that would make the stride option more discoverable?