[Bug] Unit test failed with error message "native lz4 library not available"

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.

https://paimon.apache.org/

Apache License 2.0

2.27k stars 911 forks source link

[Bug] Unit test failed with error message "native lz4 library not available" #923

Closed liming30 closed 1 year ago

liming30 commented 1 year ago

Search before asking

[X] I searched in the issues and found nothing similar.

Paimon version

0.4-snapshot

Compute Engine

flink-1.16

Minimal reproduce step

mvn test -pl paimon-format -Dtest=org.apache.paimon.format.parquet.ParquetReadWriteTest#testContinuousRepetition

What doesn't meet your expectations?

There should be no exceptions.

Anything else?

Are you willing to submit a PR?

[X] I'm willing to submit a PR!

liming30 commented 1 year ago

@Tartarus0zm Can you please help to confirm this issue? It seems to be caused by this change. When I changed it from LZ4 to SNAPPY, the error disappeared.

liming30 commented 1 year ago

@JingsongLi The compression type of LZ4 uses hadoop's native library. Should we change the compression type in the test to the default UNCOMPRESSED or other compression types?

JingsongLi commented 1 year ago

Thanks @liming30 , file compression pr reverted.