cda-group / arcon

State-first Streaming Applications in Rust
https://cda-group.github.io/arcon/
Apache License 2.0
175 stars 17 forks source link

Add Keyed/Local modes #271

Closed Max-Meldrum closed 2 years ago

Max-Meldrum commented 3 years ago

As of now everything just defaults to a keyed mode.

Set up partitioning info at the manager level and also let Nodes be aware if they are Keyed or Local.

// node level
pub enum NodeKind {
  Keyed(KeyRange),
  Local,
}
impl NodeKind {
   pub fn get_key(&self, elem: ArconElement<impl ArconType>) {
       match self {
           NodeKind::Keyed(_) => elem.data.get_key(),
           NodeKind::Local => 0, // pre-defined key....
       }
   }
}

// manager level
#[derive(Debug)]
pub struct Region {
    /// A Region may internally further split up the key ranges
    /// Example: ((0, KeyRange(0, 31), (1, KeyRange(32, 64), ....)
    ranges: Vec<(NodeID, KeyRange)>,
}
Max-Meldrum commented 2 years ago

Closing in favour of #284