search

lightbend / cloudflow

Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.
https://cloudflow.io
Apache License 2.0
321 stars 89 forks source link

Add Flink 1.12 support to Cloudflow (reason: we would like to use Azure Data Lake Store Gen2) #803

Open tringuyen-yw opened 4 years ago

tringuyen-yw commented 4 years ago

Is your feature request related to a problem? Please describe. Please add support for Flink 1.12 in Cloudflow.

Flink 1.12 add the ability to use ADLS Gen2 for checkpointing, which is very interesting for us: FLINK-18562 Add Support for Azure Data Lake Store Gen 2 in Flink File System

RayRoestenburg commented 4 years ago

As of Cloudflow 2.0.10, Flink streamlet docker images are shipped with Flink 1.11

That's not correct, at the moment we only support Flink 1.10. We would first have to see that everything works correctly with Flink 1.12. We have tried before with Flink 1.11 but there were some issues...

RayRoestenburg commented 4 years ago

(I might just be confused by the phrasing of your feature request, I guess you mean that you would like Flink 1.12 support?)

RayRoestenburg commented 4 years ago

@tringuyen-yw It looks like Flink 1.12 is not released yet: https://cwiki.apache.org/confluence/display/FLINK/1.12+Release Expected release date November?

blublinsky commented 4 years ago

I think migration to 1.11 is an important milestone, especially if we decide to use Flink kubernetes native.

synchris commented 2 years ago

Support for Flink 1.12 would be an exciting addition.