[Bug] [Connector-S3File] 事务提交时的rename，对于对象存储来说会导致严重性能问题 - Githubissues

apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

https://seatunnel.apache.org/

Apache License 2.0

8.06k stars 1.83k forks source link

[Bug] [Connector-S3File] 事务提交时的rename，对于对象存储来说会导致严重性能问题 #8047

Open dragongu opened 1 week ago

dragongu commented 1 week ago

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

OSS的rename是一个非常重的操作，会重新copy一次文件，希望能够支持直接写目的文件

SeaTunnel Version

2.3.8

SeaTunnel Config

与配置无关

Running Command

与启动命令无关

Error Exception

无异常

Zeta or Flink or Spark Version

No response

Java or Scala Version

No response

Screenshots

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct