search

python-streamz / streamz

Real-time stream processing for python
https://streamz.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
1.24k stars 148 forks source link

Time based lookback window? #466

Open anovv opened 1 year ago

anovv commented 1 year ago

Is there such a stream? I want to hold all of the events appeared within a giving time range starting from now (and do some aggregation downstream), e.g. all of the events for the last 15 mins. I found timed_window and sliding_window, but they have different purposes from what I see.

martindurant commented 1 year ago

I believe timed_window is exactly what you want. Every specified time period, it emits all the events that have been seen in that period as a tuple, and then resets its internal buffer. 

In [43]: s = streamz.Stream.from_periodic(lambda: 1, 1)

In [44]: s2 = s.timed_window(5)

In [45]: s2.sink(print)
In [46]: s.start()

[1, 1, 1, 1, 1]
[1, 1, 1, 1, 1]