unable to generate the nar file for hudi v15 with pulsar 3.3.0.5

[Pavan792reddy]

hi all, i have added the pom.xml file in the comments .

https://github.com/streamnative/pulsar-io-lakehouse/tree/v3.3.0.5

to generate the build file mvn -T 3C -e clean package -DskipTests -Pc

[WARNING] /home/pavankumar_reddy/pulsar-io-lakehouse-3.3.0.5/src/main/java/org/apache/pulsar/ecosystem/io/lakehouse/sink/hudi/HoodieWriter.java:[74,37] unchecked call to HoodieAvroRecord(org.apache.hudi.common.model.HoodieKey,T) as a member of the raw type org.apache.hudi.common.model.HoodieAvroRecord [INFO] 12 warnings [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] /home/pavankumar_reddy/pulsar-io-lakehouse-3.3.0.5/src/main/java/org/apache/pulsar/ecosystem/io/lakehouse/sink/hudi/HoodieWriterProvider.java:[47,52] incompatible types: org.apache.hadoop.conf.Configuration cannot be converted to org.apache.hudi.storage.StorageConfiguration<?> [ERROR] /home/pavankumar_reddy/pulsar-io-lakehouse-3.3.0.5/src/main/java/org/apache/pulsar/ecosystem/io/lakehouse/sink/hudi/HoodieWriterProvider.java:[57,24] incompatible types: org.apache.hadoop.conf.Configuration cannot be converted to org.apache.hudi.storage.StorageConfiguration<?> [INFO] 2 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 11.354 s (Wall Clock) [INFO] Finished at: 2024-07-24T13:51:14Z [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project pulsar-io-lakehouse: Compilation failure: Compilation failure: [ERROR] /home/pavankumar_reddy/pulsar-io-lakehouse-3.3.0.5/src/main/java/org/apache/pulsar/ecosystem/io/lakehouse/sink/hudi/HoodieWriterProvider.java:[47,52] incompatible types: org.apache.hadoop.conf.Configuration cannot be converted to org.apache.hudi.storage.StorageConfiguration<?> [ERROR] /home/pavankumar_reddy/pulsar-io-lakehouse-3.3.0.5/src/main/java/org/apache/pulsar/ecosystem/io/lakehouse/sink/hudi/HoodieWriterProvider.java:[57,24] incompatible types: org.apache.hadoop.conf.Configuration cannot be converted to org.apache.hudi.storage.StorageConfiguration<?> [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[Pavan792reddy]

here are pom.xml file details

org.apache apache 18 4.0.0 org.apache.pulsar.ecosystem pulsar-io-lakehouse 3.3.0.5 Pulsar Ecosystem :: IO Connector :: Lake House It is a lakehouse connector streaming convert data between lakehouse and Apache Pulsar. 8 8 17 UTF-8 UTF-8 true 4.2.2 0 2.13.2.1 1.18.22 3.3.0.5 2.17.2 1.7.25 3.2.4 0.13.1 1.12.0 0.15.0 4.0.2 0.3.0 1.12.2 4.1.77.Final 1.12.220 hadoop3-2.2.1 2.12.0 1.1.8.4 7.3.0 4.0.3 3.12.4 3.0 3.1.2 3.10.1 3.0.0-M3 1.2.0 8.37 4.2.2 Apache License, Version 2.0 http://www.apache.org/licenses/LICENSE-2.0.txt repo io.streamnative pulsar-io-core ${pulsar.version} com.fasterxml.jackson.core jackson-databind ${jackson.version} org.projectlombok lombok ${lombok.version} com.github.spotbugs spotbugs-annotations ${spotbugs-annotations.version} io.streamnative pulsar-client-original ${pulsar.version} provided org.apache.hadoop hadoop-client ${hadoop.version} commons-logging commons-logging log4j log4j org.apache.hadoop hadoop-yarn-* org.apache.hadoop hadoop-mapreduce-* com.sun.jersey * jetty-servlet org.eclipse.jetty jetty-util org.eclipse.jetty jetty-webapp org.eclipse.jetty io.netty netty-common ${netty.version} org.apache.logging.log4j log4j-api ${log4j2.version} org.apache.logging.log4j log4j-core ${log4j2.version} org.apache.logging.log4j log4j-1.2-api ${log4j2.version} org.apache.logging.log4j log4j-slf4j-impl ${log4j2.version} org.apache.iceberg iceberg-api ${iceberg.version} org.apache.iceberg iceberg-core ${iceberg.version} org.apache.iceberg iceberg-parquet ${iceberg.version} org.apache.iceberg iceberg-common ${iceberg.version} org.apache.iceberg iceberg-data ${iceberg.version} org.apache.iceberg iceberg-hive-metastore ${iceberg.version} org.apache.parquet parquet-avro ${parquet.avro.version} io.delta delta-standalone_2.12 ${delta.version} org.apache.parquet parquet-column ${parquet.version} org.apache.parquet parquet-hadoop ${parquet.version} org.slf4j slf4j-api ${slf4j.version} org.slf4j slf4j-simple ${slf4j.version} org.xerial.snappy snappy-java ${snappy.java.version} org.apache.hudi hudi-java-client ${hudi.version} log4j log4j org.apache.hudi hudi-io ${hudi.version} log4j log4j org.apache.hudi hudi-common ${hudi.version} log4j log4j org.apache.hudi hudi-hadoop-common ${hudi.version} log4j log4j com.esotericsoftware kryo-shaded ${kryo.version} org.apache.hadoop hadoop-aws ${hadoop.version} com.amazonaws aws-java-sdk-bundle com.amazonaws aws-java-sdk-bundle ${aws.sdk.version} com.google.cloud.bigdataoss gcs-connector ${gcs.version} shaded org.apache.hadoop hadoop-azure ${hadoop.version} jetty-util org.eclipse.jetty jetty-util-ajax org.eclipse.jetty org.testng testng ${testng.version} org.yaml * test org.awaitility awaitility ${awaitility.version} test org.mockito mockito-core ${mockito.version} test org.projectlombok lombok provided com.github.spotbugs spotbugs-annotations provided io.streamnative pulsar-io-core com.fasterxml.jackson.core jackson-databind io.streamnative pulsar-client-original provided org.apache.hadoop hadoop-client org.apache.iceberg iceberg-api org.apache.iceberg iceberg-core org.apache.iceberg iceberg-parquet org.apache.iceberg iceberg-common org.apache.iceberg iceberg-data org.apache.iceberg iceberg-hive-metastore io.delta delta-standalone_2.12 org.apache.parquet parquet-avro org.apache.parquet parquet-column org.apache.parquet parquet-hadoop org.slf4j slf4j-api org.slf4j slf4j-simple org.xerial.snappy snappy-java org.apache.logging.log4j log4j-slf4j-impl org.apache.logging.log4j log4j-api org.apache.logging.log4j log4j-core org.apache.logging.log4j log4j-1.2-api org.apache.logging.log4j log4j-slf4j-impl io.netty netty-common org.apache.hudi hudi-java-client io.prometheus * org.apache.curator * io.dropwizard.metrics * org.apache.hudi hudi-io org.apache.hudi hudi-common org.apache.hudi hudi-hadoop-common com.esotericsoftware kryo-shaded org.apache.curator curator-framework ${curator.version} log4j log4j org.apache.hudi hudi-client-common ${hudi.version} test tests log4j log4j org.testng testng test org.awaitility awaitility test org.mockito mockito-core test cloud -cloud org.apache.hadoop hadoop-aws ${hadoop.version} com.amazonaws aws-java-sdk-bundle com.amazonaws aws-java-sdk-bundle com.google.cloud.bigdataoss gcs-connector shaded org.apache.hadoop hadoop-azure jdk11 [11,) ${maven.compiler.target} --add-opens java.base/java.io=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.base/java.net=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.nio.channels.spi=ALL-UNNAMED --add-opens java.base/java.nio.file=ALL-UNNAMED --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED --add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens java.base/java.util.stream=ALL-UNNAMED --add-opens java.base/java.util.zip=ALL-UNNAMED --add-opens java.base/java.time=ALL-UNNAMED --add-opens java.base/jdk.internal.loader=ALL-UNNAMED --add-opens java.base/sun.net.dns=ALL-UNNAMED --add-opens java.base/sun.nio.ch=ALL-UNNAMED --add-opens java.base/sun.security.jca=ALL-UNNAMED --add-opens java.xml/jdk.xml.internal=ALL-UNNAMED ${project.artifactId}-${project.version}${cloudPkgSuffix} maven-compiler-plugin ${maven-compiler-plugin.version} UTF-8 -Xlint:deprecation -Xlint:unchecked -Xpkginfo:always org.apache.maven.plugins maven-surefire-plugin ${redirectTestOutputToFile} false 1800 ${testRetryCount} -Xmx2G ${test.additional.args} org.apache.nifi nifi-nar-maven-plugin ${nifi.nar.plugin.version} true ${project.artifactId}-${project.version}${cloudPkgSuffix} default-nar package nar com.mycila license-maven-plugin ${license.plugin.version}

src/license/ALv2/header.txt

LICENSE NOTICE **/*.xml .github/CODEOWNERS {project.basedir}/src/test/java/resources/external/* .github/workflows/*.yml data/** JAVADOC_STYLE SCRIPT_STYLE SCRIPT_STYLE SCRIPT_STYLE SCRIPT_STYLE XML_STYLE SCRIPT_STYLE check-license-header check process-sources org.apache.maven.plugins maven-checkstyle-plugin ${maven-checkstyle-plugin.version} com.puppycrawl.tools checkstyle ${puppycrawl.checkstyle.version} src/checkstyle/checkstyle.xml src/checkstyle/suppressions.xml UTF-8 true true false true checkstyle validate check com.github.spotbugs spotbugs-maven-plugin ${spotbugs-maven-plugin.version} ${session.executionRootDirectory}/src/spotbugs/findbugsExclude.xml org.apache.maven.plugins maven-compiler-plugin org.apache.maven.plugins maven-surefire-plugin org.apache.nifi nifi-nar-maven-plugin com.mycila license-maven-plugin org.apache.maven.plugins maven-checkstyle-plugin com.github.spotbugs spotbugs-maven-plugin

[david-streamlio]

FWIW, I was unable to reproduce this issue on my laptop using the main branch.

[SimplytheVinay]

FWIW, I was unable to reproduce this issue on my laptop using the main branch.

@david-streamlio : Check this issue. Same problem: https://github.com/streamnative/pulsar-io-lakehouse/issues/645