search

apache / iceberg

Apache Iceberg
https://iceberg.apache.org/
Apache License 2.0
6.39k stars 2.21k forks source link

com/fasterxml/jackson/core/exc/StreamConstraintsException #9355

Open smileyboy2019 opened 10 months ago

smileyboy2019 commented 10 months ago

Apache Iceberg version

None

Query engine

None

Please describe the bug 🐞

I used the iceberg 1.4.2 Java API to create a table, which can be created, but an error occurred when checking the table properties. Both table. local() and select will report errors. The first create does not report an error.

Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/exc/StreamConstraintsException at com.fasterxml.jackson.databind.node.JsonNodeFactory.objectNode(JsonNodeFactory.java:353) at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:100) at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:25) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4825) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3809) at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:273) at org.apache.iceberg.TableMetadataParser.read(TableMetadataParser.java:266) at org.apache.iceberg.hadoop.HadoopTableOperations.updateVersionAndMetadata(HadoopTableOperations.java:98) at org.apache.iceberg.hadoop.HadoopTableOperations.refresh(HadoopTableOperations.java:121) at org.apache.iceberg.hadoop.HadoopTableOperations.current(HadoopTableOperations.java:84) at org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:189) at org.apache.iceberg.catalog.Catalog.createTable(Catalog.java:75) at org.apache.iceberg.catalog.Catalog.createTable(Catalog.java:106) at IcebergExample.main(IcebergExample.java:58) Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.exc.StreamConstraintsException at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351)

nastra commented 10 months ago

@smileyboy2019 this seems to be similar to https://github.com/apache/iceberg/issues/9339. You're missing a dependency at runtime. Can you please post the dependencies you're using? We normally package all required dependencies into a runtime jar e.g. for Spark: iceberg-spark-runtime-3.5_2.12.

Given that you're selectively using only a handful of dependencies, you need to make sure that you add the right dependencies that iceberg requires.

smileyboy2019 commented 10 months ago
gbicc.net 
<artifactId>data-lake</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>8</source>
                <target>8</target>
            </configuration>
        </plugin>
    </plugins>
</build>

<dependencies>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-core_2.13</artifactId>
        <version>3.5.0</version>
    </dependency>
    <dependency>
        <groupId>io.delta</groupId>
        <artifactId>delta-spark_2.12</artifactId>
        <version>3.0.0</version>
    </dependency>
    <!--  https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-core  -->
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-core</artifactId>
        <version>1.4.2</version>
    </dependency>

    <dependency>
        <groupId>io.minio</groupId>
        <artifactId>minio</artifactId>
        <version>8.5.7</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-s3 -->
    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk-s3</artifactId>
        <version>1.12.620</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-aws</artifactId>
        <version>3.2.2</version>
    </dependency>

    <dependency>
        <groupId>org.apache.parquet</groupId>
        <artifactId>parquet-hadoop</artifactId>
        <version>1.10.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-api</artifactId>
        <version>1.4.2</version>
    </dependency>
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-bundled-guava</artifactId>
        <version>1.4.2</version>

    </dependency>
    <!-- 其他依赖库 -->
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-core</artifactId>
        <version>2.12.4</version>
    </dependency>
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-common</artifactId>
        <version>1.4.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-parquet -->
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-parquet</artifactId>
        <version>1.4.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-aws -->
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-aws</artifactId>
        <version>1.4.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-common -->
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-common</artifactId>
        <version>1.4.2</version>
        <scope>runtime</scope>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-arrow -->
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-arrow</artifactId>
        <version>1.4.2</version>
    </dependency>
    <!-- https://mvnrepository.com/artifact/org.apache.iceberg/iceberg-data -->
    <dependency>
        <groupId>org.apache.iceberg</groupId>
        <artifactId>iceberg-data</artifactId>
        <version>1.4.2</version>
    </dependency>
</dependencies>
github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.