apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.43k stars 954 forks source link

[Bug] Caused by: java.lang.NoClassDefFoundError: Could not initialize class org.apache.paimon.shade.com.github.luben.zstd.ZstdInputStreamNoFinalizer #4531

Open ChrisKyrie opened 1 week ago

ChrisKyrie commented 1 week ago

Search before asking

Paimon version

0.9

Compute Engine

Hive 3.1.3

Minimal reproduce step

I encountered a problem when I read the Paimon table with Hive。${HIVE_HOME}/lib has zstd-jni-1.3.2-2.jar,Paimon depend zstd-jni-1.5.5-11.jar,I try to shade zstd in paimon-format.xml.

<relocation>
    <pattern>com.github.luben</pattern>
    <shadedPattern>org.apache.paimon.shade.com.github.luben</shadedPattern>
</relocation>

But the exception occurred java.lang.NoClassDefFoundError: Could not initialize class org.apache.paimon.shade.com.github.luben.zstd.ZstdInputStreamNoFinalizer.

I made sure I added the jar to the classpath。Does zstd-jni not allow shade?

What doesn't meet your expectations?

java.lang.NoClassDefFoundError: Could not initialize class org.apache.paimon.shade.com.github.luben.zstd.ZstdInputStreamNoFinalizer.

Anything else?

No response

Are you willing to submit a PR?

ChrisKyrie commented 1 week ago

If I do not shade zstd-jni,I had a paroblem like this. I don't want to modify the zstd-jni version under hive, I want to use shade https://github.com/apache/paimon/issues/3492

Chouving77 commented 11 hours ago

I also tried to shade the zstd, but I found that after shading it seems that it cannot find the native lib