[Feature] Support inferring scan parallelism for source in flink

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.29k stars 916 forks source link

[Feature] Support inferring scan parallelism for source in flink #1943

Closed FangYongs closed 1 year ago

FangYongs commented 1 year ago

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation

Currently Paimon only infer scan parallelism from options in table for flink jobs, it's better to support this when the option is configured in flink. subtask of #1945

Solution

Support inferring scan parallelism in flink

Anything else?

No response

Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

FangYongs commented 1 year ago

The default value is true now, we need to disable this flag for flink job dynamic option