search

flock-lab / flock

Flock: A Low-Cost Streaming Query Engine on FaaS Platforms
https://flock-lab.github.io/flock/
GNU Affero General Public License v3.0
285 stars 39 forks source link

fix: add `rand u128` to payload's `tid` #395

Closed gangliao closed 2 years ago

gangliao commented 2 years ago

Which issue does this PR close?

This PR can address the session window issue.

Closes #394

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

By submitting this pull request

gangliao commented 2 years ago 

START RequestId: 58921d50-71a8-4d3e-a092-3e3d28d38953 Version: $LATEST
--
[2021-12-30T19:46:38Z INFO  flock::datasource::nexmark::nexmark] Generating events for 30s over 1 partitions.
[2021-12-30T19:46:39Z INFO  flock::nexmark::source] Nexmark Benchmark: Query 11
[2021-12-30T19:46:39Z INFO  flock::nexmark::source] NEXMarkSource { config: Config { args: {"events-per-second": "1000", "threads": "1", "seconds": "30"} }, window: SessionWindow(Seconds(10)) }
[2021-12-30T19:46:39Z INFO  flock::nexmark::source] [OK] Generate nexmark events.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 0: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 1: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 2: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 3: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 4: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 5: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 6: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 7: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 8: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 9: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 10: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 11: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 12: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 13: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 14: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 15: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 16: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 17: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 18: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 19: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 20: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 21: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 22: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 23: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 24: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 25: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:39Z INFO  flock::window] Session window -> function name: q11-00-05
[2021-12-30T19:46:39Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-05.
[2021-12-30T19:46:39Z INFO  flock::window] [OK] Event 0 - function payload bytes: 1779
[2021-12-30T19:46:40Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 26: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:40Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 27: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:40Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 28: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:40Z INFO  flock::window] Session window -> function name: q11-00-16
[2021-12-30T19:46:40Z INFO  flock::window] Session window -> function name: q11-00-31
[2021-12-30T19:46:40Z INFO  flock::window] Session window -> function name: q11-00-11
[2021-12-30T19:46:40Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-31.
[2021-12-30T19:46:40Z INFO  flock::window] [OK] Event 0 - function payload bytes: 1564
[2021-12-30T19:46:40Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-16.
[2021-12-30T19:46:40Z INFO  flock::window] [OK] Event 0 - function payload bytes: 2289
[2021-12-30T19:46:40Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-11.
[2021-12-30T19:46:40Z INFO  flock::window] [OK] Event 0 - function payload bytes: 2840
[2021-12-30T19:46:41Z INFO  flock::datasource::nexmark::nexmark] Selecting events for epoch 29: 20 persons, 60 auctions, 920 bids.
[2021-12-30T19:46:41Z INFO  flock::window] Session window -> function name: q11-00-14
[2021-12-30T19:46:41Z INFO  flock::window] Session window -> function name: q11-00-04
[2021-12-30T19:46:41Z INFO  flock::window] Session window -> function name: q11-00-24
[2021-12-30T19:46:41Z INFO  flock::window] Session window -> function name: q11-00-00
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-14.
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Event 0 - function payload bytes: 1710
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-04.
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Event 0 - function payload bytes: 2193
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-24.
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Event 0 - function payload bytes: 1730
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Send 1 events from a session window to function: q11-00-00.
[2021-12-30T19:46:41Z INFO  flock::window] [OK] Event 0 - function payload bytes: 2064
END RequestId: 58921d50-71a8-4d3e-a092-3e3d28d38953
REPORT RequestId: 58921d50-71a8-4d3e-a092-3e3d28d38953  Duration: 3692.59 ms    Billed Duration: 3786 ms    Memory Size: 2048 MB    Max Memory Used: 54 MB  Init Duration: 92.53 ms
gangliao commented 2 years ago

Now, consistent hashing works perfectly.