cda-group / arcon

State-first Streaming Applications in Rust
https://cda-group.github.io/arcon/
Apache License 2.0
175 stars 17 forks source link

Rolling Aggregations #313

Open Max-Meldrum opened 2 years ago

Max-Meldrum commented 2 years ago

While it is possible to implement by hand, provide a way to express rolling aggregations in the API.

#[arcon::proto]
#[derive(Arcon, Copy, Clone)]
#[arcon(unsafe_ser_id = 12, reliable_ser_id = 13, version = 1)]
pub struct Event {
    pub id: u64,
    pub data: u64,
}

let mut app = Application::default()
   .iterator((0..1000000).map(|x| Event { id: x, data: 1.5 }), |conf| {
         conf.set_timestamp_extractor(|x: &Event| x.id);
   })
  .key_by(|event: &Event| &event.id)
  .sum(|event: Event|  event.data) // output: (key, current_sum)
  .build();