confluentinc / event-streaming-patterns

A collection of Event Streaming Patterns, including problem statements, solutions, and implementation examples.

Apache License 2.0

13 stars 6 forks source link

Pattern for Timestamp assigner #145

Closed bbejeck closed 3 years ago

bbejeck commented 3 years ago

Description

115

New Pattern Checklist

[x] Pattern file name should follow docs/pattern-category/pattern-name.md
[x] Create an on-brand diagram (https://docs.google.com/presentation/d/1Zf256Z6fBvre3uclIbmxXsDpnTIxiBX66b13pHbGIYc/edit#slide=id.gdadc7bce69_4_0)
[x] Implement ksqlDB or code example (not all patterns will have code, but highly desired)
[x] Provide references
[x] Validate hyperlinks
[x] Spell check (e.g. using aspell)

benstopford commented 3 years ago

Two things we might want to mention here are: timestamps are used to drive processing in some stream processors. also that timestamps are used to determine out-of-order data and window assignment. I think the latter seems particularly important although you likely understand this better than I. Maybe mention in the preamble prior to the problem statement?

miguno commented 3 years ago

Two things we might want to mention here are: timestamps are used to drive processing in some stream processors. also that timestamps are used to determine out-of-order data and window assignment. I think the latter seems particularly important although you likely understand this better than I. Maybe mention in the preamble prior to the problem statement?

Regarding such a preamble: take a look at the suggested text I added to https://github.com/confluentinc/event-streaming-patterns/pull/45.

bbejeck commented 3 years ago

@ybyzek thanks for the review updated per comments

@benstopford, @miguno I've updated this to use the preamble from #45

bbejeck commented 3 years ago

Updated per comments.

rspurgeon commented 3 years ago

Screen Shot 2021-06-09 at 1 40 22 PM

@bbejeck for your consideration I made these adjustments to the diagram

bbejeck commented 3 years ago

@rspurgeon I've updated the implementation section per offline discussion.

\cc @miguno

bbejeck commented 3 years ago

Based on an offline conversation with @miguno we're dropping the timestamp-assigner pattern and instead will describe the timestamp assignment process in the event-time-processing and wallclock-time-processing patterns.