search

getindata / flink-http-connector

Http Connector for Apache Flink. Provides sources and sinks for Datastream , Table and SQL APIs.
Apache License 2.0
149 stars 42 forks source link

Support Flink 2 (and preview) #126

Open davidradl opened 1 day ago

davidradl commented 1 day ago

There may be a different java version. Also there is a Flink lineage tracing capability that we may want to take advantage of. The dev list has a post proposing lots of deliverables for v1 pre lineage and V1 with lineage and V2 Kafka connector.

davidradl commented 1 day ago

I am happy to work on this. @kzarzycki

kzarzycki commented 1 day ago

Sure @davidradl ! If you needed help with lineage APIs and/or OpenLineage, our colleagues from Getindata actually implemented it, like @pawel-big-lebowski :) I will also help with getting reviewers for that.

davidradl commented 1 day ago

Sure @davidradl ! If you needed help with lineage APIs and/or OpenLineage, our colleagues from Getindata actually implemented it, like @pawel-big-lebowski :) I will also help with getting reviewers for that.

sounds good. I will see how the Kafka connector plays out, in terms of java levels. before looking at this.

pawel-big-lebowski commented 9 hours ago

@davidradl I think we should be ready soon with the first version of job listener that converts Flink lineage into OpenLineage events. We'll bring more details during Flink Forward talk https://www.flink-forward.org/berlin-2024/agenda#data-lineage-for-apache-flink-with-openlineage

davidradl commented 5 hours ago

@davidradl I think we should be ready soon with the first version of job listener that converts Flink lineage into OpenLineage events. We'll bring more details during Flink Forward talk https://www.flink-forward.org/berlin-2024/agenda#data-lineage-for-apache-flink-with-openlineage

sounds good - I am attending so will come along and say hello!