search

apache / incubator-streampark

Make stream processing easier! Easy-to-use streaming application development framework and operation platform.
https://streampark.apache.org/
Apache License 2.0
3.82k stars 983 forks source link

[Bug] yarn application mode fail to submit job(s3p checkpoint) #2895

Open ccpypy opened 1 year ago

ccpypy commented 1 year ago

Search before asking

Java Version

11

Scala Version

2.12.x

StreamPark Version

2.1.1

Flink Version

1.13.5

deploy mode

yarn-application

What happened

when i use common-cli to submit job,it's ok flink run -t yarn-per-job \ -Dyarn.application.name=cdcsyncworkorder \ -Djobmanager.memory.process.size=1024MB \ -Dtaskmanager.memory.process.size=1024mb \ -Dtaskmanager.memory.managed.size=0MB \ -Dtaskmanager.numberOfTaskSlots=1 \ --fromSavepoint s3p://xxxxxxx

but i submit job by streampark(yarn-application),start application with checkpoint, throwing error.

Error Exception

Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 's3p'. The scheme is directly supported by Flink through the following plugin: flink-s3-fs-presto. Please ensure that each plugin resides within its own subfolder within the plugins directory. See https://ci.apache.org/projects/flink/flink-docs-stable/ops/plugins.html for more information. If you want to use a Hadoop file system for that scheme, please add the scheme to the configuration fs.allowed-fallback-filesystems. For a full list of supported file systems, please see https://nightlies.apache.org/flink/flink-docs-stable/ops/filesystems/.

No response

Screenshots

No response

Are you willing to submit PR?

Code of Conduct

ccpypy commented 1 year ago

Need to upload related jar to the HDFS directory