search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.27k stars 911 forks source link

[Bug] PluginLoader should use yarn tmp directory for yarn cluster #933

Closed JingsongLi closed 1 year ago

JingsongLi commented 1 year ago

Search before asking

Paimon version

Take a look to System.getenv("LOCAL_DIRS"), if it is not null, use it.

Compute Engine

flink

Minimal reproduce step

run paimon-flink with yarn cluster.

What doesn't meet your expectations?

tmp directory generates a large number of files

Anything else?

No response

Are you willing to submit a PR?

calvinjiang commented 1 year ago

I'll try to check and fix this issue.