apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
13.87k stars 3.38k forks source link

[Java] Jni mvn generate-resources failed because not generate arrow-bom #41808

Open jinchengchenghh opened 1 month ago

jinchengchenghh commented 1 month ago

Describe the bug, including details regarding any error messages, version, and platform.

arrow_ep/src/arrow_ep/java# mvn generate-resources -P generate-libs-cdata-all-os -Darrow.c.jni.dist.dir=$ARROW_INSTALL_DIR       -Dmaven.test.skip -Drat.skip -Dmaven.gitcommitid.skip -Dcheckstyle.skip -N
[INFO] Scanning for projects...
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/arrow/arrow-bom/15.0.0-gluten-3/arrow-bom-15.0.0-gluten-3.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Non-resolvable import POM: The following artifacts could not be resolved: org.apache.arrow:arrow-bom:pom:15.0.0-gluten-3 (absent): Could not find artifact org.apache.arrow:arrow-bom:pom:15.0.0-gluten-3 in central (https://repo.maven.apache.org/maven2) @ line 601, column 20
 @
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project org.apache.arrow:arrow-java-root:15.0.0-gluten-3 (/mnt/DP_disk1/code/incubator-gluten/ep/build-velox/build/velox_ep/_build/release/third_party/arrow_ep/src/arrow_ep/java/pom.xml) has 1 error
[ERROR]     Non-resolvable import POM: The following artifacts could not be resolved: org.apache.arrow:arrow-bom:pom:15.0.0-gluten-3 (absent): Could not find artifact org.apache.arrow:arrow-bom:pom:15.0.0-gluten-3 in central (https://repo.maven.apache.org/maven2) @ line 601, column 20 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException

Component(s)

Java

vibhatha commented 1 month ago

@jinchengchenghh could we get more context here? I assume this is building from source?

jinchengchenghh commented 1 month ago

I cannot follow the guide to build jni libraries. The command is mvn generate-resources -Pgenerate-libs-jni-macos-linux -N

https://arrow.apache.org/docs/developers/java/building.html#id14

jduo commented 1 month ago

Hi @jinchengchenghh , just wondering, are you running the mvn generate-resources command before doing a mvn install?

This looks as though you've created a new arrow version 15.0.0-gluten-3, so maven can't find the BOM artifact in a remote repo, and if you haven't built arrow using mvn install, it wouldn't be found in your local repo.

jinchengchenghh commented 1 month ago

Yes, I thinks it is not helpful, the main branch is xx-SNAPSHOT, we cannot also build like before. Yes, we should run mvn generate-resources to generate .so file, and then use mvn install to generate jar file which include the existed .so file. I can run mvn clean install and then mvn generate-resource and then mvn install though it looks rediculous. It still has a shortage, I don't need to build many dependencies like flightsql before, but now I need to build many modules included in arrow-bom