apache / incubator-streampark

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
https://streampark.apache.org/
Apache License 2.0
3.84k stars 990 forks source link

[Bug] Compatibility problem under fink 1.17.0 version when restoring job from savepoint #2723

Closed cygnusdark closed 9 months ago

cygnusdark commented 1 year ago

Search before asking

Java Version

1.8

Scala Version

2.12.x

StreamPark Version

2.1.0

Flink Version

1.17.0

deploy mode

yarn-application

What happened

When I restart a job from previous savepoint, I got an error exception.

It looks like a compatibility problem.

It seems like we need to upgrade commons-cli version to 1.5, or upgrade the Flink version in Flink shims base

Is there better way to fix this problem?

Error Exception

Caused by: java.lang.NoSuchMethodError: org.apache.commons.cli.CommandLine.hasOption(Lorg/apache/commons/cli/Option;)Z
    at org.apache.flink.client.cli.CliFrontendParser.createSavepointRestoreSettings(CliFrontendParser.java:631)
    at org.apache.flink.client.cli.ProgramOptions.<init>(ProgramOptions.java:119)
    at org.apache.flink.client.cli.ProgramOptions.create(ProgramOptions.java:192)
    at org.apache.streampark.flink.client.trait.FlinkClientTrait.submit(FlinkClientTrait.scala:87)
    at org.apache.streampark.flink.client.trait.FlinkClientTrait.submit$(FlinkClientTrait.scala:63)
    at org.apache.streampark.flink.client.impl.YarnApplicationClient$.submit(YarnApplicationClient.scala:44)
    at org.apache.streampark.flink.client.FlinkClientHandler$.submit(FlinkClientHandler.scala:40)
    at org.apache.streampark.flink.client.FlinkClientHandler.submit(FlinkClientHandler.scala)

Screenshots

image

Are you willing to submit PR?

Code of Conduct

wolfboys commented 1 year ago

Our developer has tested it and found no issues running it under Flink 1.17. You may want to check if there are any commons-cli related jars in your flink/lib directory.

wolfboys commented 9 months ago

Answered

Level1Accelerator commented 1 month ago

Answered

同样碰到了这个问题 Scala Version 2.12.x

StreamPark Version 2.1.3

Flink Version 1.18.1 我也往flink lib下放了commons-cli 1.5的包,还是这个错