Open mbroecheler opened 11 months ago
Not sure if this is something we can "fix" as it is Flink behavior, but we should investigate if we can detect this case and warn about it or use a different watermarking strategy for this in our examples so people don't run into it.
Let's add an warning when mixing bounded with unbounded streams if the file monitor config value isn't set.
For example, when running the following script (on the loan data example):
where the
Applications
source reads from local file continuously (i.e."monitorIntervalMs" : "0"
is NOT set),AppCount
will be empty in the database. However,Applications
are written to the database.I assume this is a watermarking issue since the watermark does not advance, the aggregation never fires.