composewell / streamly

High performance, concurrent functional programming abstractions
https://streamly.composewell.com
Other
857 stars 64 forks source link

Generalized representation of a sliding window #2158

Open harendra-kumar opened 2 years ago

harendra-kumar commented 2 years ago

Currently we are using (a, Maybe a) type for a window operation. It allows growing the window using (a, Nothing) and sliding the window using (a, Just a). However for a fully general window we may also need a window shrinking operation which could be (Just a, a). So the general type would be (Maybe a, Maybe a). However we can simplify it to three constructors:

data Window a = 
      Grow a
    | Shrink a
    | Slide a a

Existing folds would have to be modified to handle the shrink operation.

harendra-kumar commented 2 years ago

Primarily, I wanted a flat type instead of (a, Maybe a) which has one level of nesting of constructors. The Shrink constructor was introduced for completeness.

But it could be quite useful e.g. if we are computing stats over last one hour of trades. The window would be growing when trade frequency is increasing, the window would remain constant when the trade frequency is steady, but it would shrink when the trade frequency reduces. If no trades are happening our clock would still be ticking and to maintain a 1 hour window we would be ejecting the oldest elements from the window even without any other elements entering the window. In fact, it is required for time based windows.

harendra-kumar commented 2 years ago

Or we could name these as Insert/Delete/Replace. Slide or Replace is nothing but a combination of Insert and Delete, but can perhaps be implemented more efficiently in some cases.

harendra-kumar commented 1 year ago
harendra-kumar commented 1 month ago

The new Scanl module can try this so that we do not make a breaking change later.