apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.78k stars 1.74k forks source link

[Feature]Parquet version upgrade to support LZ4_RAW compression format #7223

Open liunaijie opened 1 month ago

liunaijie commented 1 month ago

Search before asking

Description

Now, we use parquet-avro 1.12.3, in this version it not support read LZ4_RAW format file. if we want support this format, we at least upgrade it to 1.13.0.

https://github.com/apache/parquet-java/blob/apache-parquet-1.12.3/parquet-common/src/main/java/org/apache/parquet/hadoop/metadata/CompressionCodecName.java

https://github.com/apache/parquet-java/blob/apache-parquet-1.13.0/parquet-common/src/main/java/org/apache/parquet/hadoop/metadata/CompressionCodecName.java

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

Code of Conduct

liunaijie commented 1 month ago

image

also need add more data type support, like UINT8, UINT16, UINT32

github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] commented 1 week ago

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

fcb-xiaobo commented 1 week ago

please assign to me