search

streamnative / pulsar-archived

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org
Apache License 2.0
73 stars 25 forks source link

ISSUE-5866: Flink event time based windowing functions not working with pulsar #19

Closed sijie closed 4 years ago

sijie commented 4 years ago

Original Issue: apache/pulsar#5866

We are using Event time based windowing functions in Flink. The source is a partitioned Pulsar topic . The timestamp field is decided by the processing function in flink. The event time is extracted from the message by attaching a TimeStampExtractor to the stream which extracts time from the message

Expected behavior

The window should get triggered once the watermark is crossed.

Actual behavior

We noticed that the window does not triggered though the watermark is crossed.

Steps to reproduce

Attaching the sample code. pulsar-flink-preload-new.zip

System configuration

Pulsar version: 2.4.2 Flink version : 1.8.2

yjshen commented 4 years ago

As in the origin issue, I've provided a flink-pulsar test framework in the thread and ask him to do some explorations by debugging the flink program.

This issue is not much Pulsar connector related but more a problematic flink program. I'd like to move this to backlog or mark it as done. @sijie thoughts?

sijie commented 4 years ago

@yjshen okay let's move it to backlog