search

dmlc / xgboost

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
https://xgboost.readthedocs.io/en/stable/
Apache License 2.0
26.29k stars 8.73k forks source link

[jvm-packages]Error in running java Demo code #1322

Closed zig-kwin-hu closed 8 years ago

zig-kwin-hu commented 8 years ago

I am running the java Demo code copied from https://github.com/dmlc/xgboost/blob/master/jvm-packages/xgboost4j-example/src/main/java/ml/dmlc/xgboost4j/java/example/BasicWalkThrough.java But an error is given. The error information appears:

七月 02, 2016 10:46:22 下午 ml.dmlc.xgboost4j.java.NativeLibLoader smartLoad 严重: failed to load library from both native path and jar 七月 02, 2016 10:46:22 下午 ml.dmlc.xgboost4j.java.DMatrix 严重: load native library failed. 七月 02, 2016 10:46:22 下午 ml.dmlc.xgboost4j.java.DMatrix 严重: java.io.FileNotFoundException: File /lib/libxgboost4j.dylib was not found inside JAR. Exception in thread "main" java.lang.UnsatisfiedLinkError: ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Ljava/lang/String;I[J)I at ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Native Method) at ml.dmlc.xgboost4j.java.DMatrix.(DMatrix.java:83) at newtest.mytest.main(mytest.java:27)

the code is :

package newtest; import java.io.File; import java.io.IOException; import java.util.Arrays; import java.util.HashMap; import ml.dmlc.xgboost4j.java.Booster; import ml.dmlc.xgboost4j.java.DMatrix; import ml.dmlc.xgboost4j.java.XGBoost; import ml.dmlc.xgboost4j.java.XGBoostError; //import ml.dmlc.xgboost4j.java.example.util.DataLoader; public class mytest { public static boolean checkPredicts(float[][] fPredicts, float[][] sPredicts) { if (fPredicts.length != sPredicts.length) { return false; }

    for (int i = 0; i < fPredicts.length; i++) {
      if (!Arrays.equals(fPredicts[i], sPredicts[i])) {
        return false;
      }
    }

    return true;
  }
public static void main(String []  args) throws XGBoostError{
    System.out.println("test");
    DMatrix trainMat = new DMatrix("/Users/huzikun/xgboost/demo/data/agaricus.txt.train");
    DMatrix testMat = new DMatrix("/Users/huzikun/xgboost/demo/data/agaricus.txt.test");

    HashMap<String, Object> params = new HashMap<String, Object>();
    params.put("eta", 1.0);
    params.put("max_depth", 2);
    params.put("silent", 1);
    params.put("objective", "binary:logistic");

    HashMap<String, DMatrix> watches = new HashMap<String, DMatrix>();
    watches.put("train", trainMat);
    watches.put("test", testMat);

    //set round
    int round = 2;

    //train a boost model
    Booster booster = XGBoost.train(trainMat, params, round, watches, null, null);

    //predict
    float[][] predicts = booster.predict(testMat);

    //save model to modelPath
    File file = new File("./model");
    if (!file.exists()) {
      file.mkdirs();
    }

    String modelPath = "./model/xgb.model";
    booster.saveModel(modelPath);

    //dump model
    booster.getModelDump("./model/dump.raw.txt", false);

    //dump model with feature map
    booster.getModelDump("../../demo/data/featmap.txt", false);

    //save dmatrix into binary buffer
    testMat.saveBinary("./model/dtest.buffer");

    //reload model and data
    Booster booster2 = XGBoost.loadModel("./model/xgb.model");
    DMatrix testMat2 = new DMatrix("./model/dtest.buffer");
    float[][] predicts2 = booster2.predict(testMat2);

    //check the two predicts
    System.out.println(checkPredicts(predicts, predicts2));

    System.out.println("start build dmatrix from csr sparse data ...");
    //build dmatrix from CSR Sparse Matrix
    /*DataLoader.CSRSparseData spData;

        spData = DataLoader.loadSVMFile("/Users/huzikun/xgboost/demo/data/agaricus.txt.train");

    DMatrix trainMat2 = new DMatrix(spData.rowHeaders, spData.colIndex, spData.data,
            DMatrix.SparseType.CSR);
    trainMat2.setLabel(spData.labels);

    //specify watchList
    HashMap<String, DMatrix> watches2 = new HashMap<String, DMatrix>();
    watches2.put("train", trainMat2);
    watches2.put("test", testMat2);
    Booster booster3 = XGBoost.train(trainMat2, params, round, watches2, null, null);
    float[][] predicts3 = booster3.predict(testMat2);

    //check predicts
    System.out.println(checkPredicts(predicts, predicts3));*/
    System.out.println("fuckyou");
}

}

It appears that the error happens when this line is running: DMatrix trainMat = new DMatrix("/Users/huzikun/xgboost/demo/data/agaricus.txt.train"); DMatrix testMat = new DMatrix("/Users/huzikun/xgboost/demo/data/agaricus.txt.test");

This problem has bothered me for two days. I would appreciate it if anyone could help. Thanks a lot.

CodingCat commented 8 years ago

have you build the library successfully by running mvn package?

zig-kwin-hu commented 8 years ago

After running mvn package under ivm-packages, it shows error information as follows: ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project xgboost4j: There are test failures. [ERROR]

So I run mvn -DskipTests=true package. And get xgboos4j-0.5.jar file in jam-package/x4jboost/target. I use it in my DEMO.java's lib. But it doesn't work.

CodingCat commented 8 years ago

you have to ensure that you can pass the unit test before you use it in your proj.....so when you found some error in your local side, you'd find out the reason such as native lib did not get compiled successfully instead of skipping

zig-kwin-hu commented 8 years ago

Set my JAVA_HOME as /Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home. So I suppose I have set it correctly. I couldn't find out why I can run mvn package successfully. Is there any guide to follow under such situation?

CodingCat commented 8 years ago

sorry....I didn't understand your comments....

zig-kwin-hu commented 8 years ago

I'm sorry that I didn't make it clear. After running mvn package under jam-packages, I see error information below: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project xgboost4j: I don't know what does this error information means.

CodingCat commented 8 years ago

would you post complete output in your screen?

zig-kwin-hu commented 8 years ago

The complete output after running mvn package is as follow: [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for ml.dmlc:xgboost4j:jar:0.5 [WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:exec-maven-plugin is missing. @ line 32, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] xgboost-jvm [INFO] xgboost4j [INFO] xgboost4j-spark [INFO] xgboost4j-flink [INFO] xgboost4j-example [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building xgboost-jvm 0.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- scalastyle-maven-plugin:0.8.0:check (checkstyle) @ xgboost-jvm --- [WARNING] sourceDirectory is not specified or does not exist value=/Users/huzikun/xgboost/jvm-packages/src/main/scala [WARNING] testSourceDirectory is not specified or does not exist value=/Users/huzikun/xgboost/jvm-packages/src/test/scala Processed 0 file(s) Found 0 errors Found 0 warnings Found 0 infos Finished in 68 ms [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ xgboost-jvm --- [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (compile) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.2:testCompile (test-compile) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ xgboost-jvm --- Discovery starting. Discovery completed in 39 milliseconds. Run starting. Expected test count is: 0 DiscoverySuite: Run completed in 97 milliseconds. Total number of tests run: 0 Suites: completed 1, aborted 0 Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 No tests were executed. [INFO] [INFO] --- maven-assembly-plugin:2.6:single (make-assembly) @ xgboost-jvm --- [INFO] Assemblies have been skipped per configuration of the skipAssembly parameter. [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building xgboost4j 0.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- scalastyle-maven-plugin:0.8.0:check (checkstyle) @ xgboost4j --- Processed 7 file(s) Found 0 errors Found 0 warnings Found 0 infos Finished in 448 ms [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ xgboost4j --- [INFO] [INFO] --- exec-maven-plugin:1.5.0:exec (native) @ xgboost4j --- build java wrapper c++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -I/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/include -I./java -I/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/include/darwin -shared -o jvm-packages/lib/libxgboost4j.so jvm-packages/xgboost4j/src/native/xgboost4j.cpp build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp In file included from jvm-packages/xgboost4j/src/native/xgboost4j.cpp:16: In file included from include/xgboost/base.h:10: dmlc-core/include/dmlc/omp.h:13:9: warning: Warning: OpenMP is not available, project will be compiled into single-thread code. Use OpenMP-enabled compiler to get benefit of multi-threading. [-W#pragma-messages]

pragma message("Warning: OpenMP is not available, " \

    ^

1 warning generated. ld: library not found for -lgomp clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *\ [jvm-packages/lib/libxgboost4j.so] Error 1 move native lib mv: rename lib/libxgboost4j.so to xgboost4j/src/main/resources/lib/libxgboost4j.dylib: No such file or directory complete [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ xgboost4j --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ xgboost4j --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (compile) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ xgboost4j --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/huzikun/xgboost/jvm-packages/xgboost4j/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ xgboost4j --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- scala-maven-plugin:3.2.2:testCompile (test-compile) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ xgboost4j ---

T E S T S

Running TestSuite [ERROR]: No test suite found. Nothing to run org.testng.TestNGException: Failure in JUnit mode for class ml.dmlc.xgboost4j.java.BoosterImplTest: could not create/run JUnit test suite: cannot retrieve JUnit method at org.testng.junit.JUnitTestRunner.runFailed(JUnitTestRunner.java:230) at org.testng.junit.JUnitTestRunner.start(JUnitTestRunner.java:223) at org.testng.junit.JUnitTestRunner.run(JUnitTestRunner.java:204) at org.testng.TestRunner$2.run(TestRunner.java:639) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRunJUnit(TestRunner.java:664) at org.testng.TestRunner.run(TestRunner.java:576) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:198) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) org.testng.TestNGException: Failure in JUnit mode for class ml.dmlc.xgboost4j.java.DMatrixTest: could not create/run JUnit test suite: cannot retrieve JUnit method at org.testng.junit.JUnitTestRunner.runFailed(JUnitTestRunner.java:230) at org.testng.junit.JUnitTestRunner.start(JUnitTestRunner.java:223) at org.testng.junit.JUnitTestRunner.run(JUnitTestRunner.java:204) at org.testng.TestRunner$2.run(TestRunner.java:639) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRunJUnit(TestRunner.java:664) at org.testng.TestRunner.run(TestRunner.java:576) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:198) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.281 sec - in TestSuite

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ xgboost4j --- Discovery starting. Discovery completed in 98 milliseconds. Run starting. Expected test count is: 7 ScalaBoosterImplSuite: 七月 07, 2016 12:09:29 上午 ml.dmlc.xgboost4j.java.NativeLibLoader smartLoad 严重: failed to load library from both native path and jar 七月 07, 2016 12:09:29 上午 ml.dmlc.xgboost4j.java.DMatrix 严重: load native library failed. 七月 07, 2016 12:09:29 上午 ml.dmlc.xgboost4j.java.DMatrix 严重: java.io.FileNotFoundException: File /lib/libxgboost4j.dylib was not found inside JAR. * RUN ABORTED * java.lang.UnsatisfiedLinkError: ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Ljava/lang/String;I[J)I at ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Native Method) at ml.dmlc.xgboost4j.java.DMatrix.(DMatrix.java:83) at ml.dmlc.xgboost4j.scala.DMatrix.(DMatrix.scala:31) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply$mcV$sp(ScalaBoosterImplSuite.scala:81) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply(ScalaBoosterImplSuite.scala:80) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply(ScalaBoosterImplSuite.scala:80) at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) ... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] xgboost-jvm ........................................ SUCCESS [ 2.374 s] [INFO] xgboost4j .......................................... FAILURE [ 3.378 s] [INFO] xgboost4j-spark .................................... SKIPPED [INFO] xgboost4j-flink .................................... SKIPPED [INFO] xgboost4j-example .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.866 s [INFO] Finished at: 2016-07-07T00:09:29+08:00 [INFO] Final Memory: 23M/457M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project xgboost4j: There are test failures -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :xgboost4j

CodingCat commented 8 years ago

so, it says

ld: library not found for -lgomp clang: error: linker command failed with exit code 1 (use -v to see invocation)

run export CXX=clang-omp++ then compile

zig-kwin-hu commented 8 years ago

Thanks for advice. After run export CXX=clang-omp++, I run mvn compile, it succeed. But then I run mvn package, it still failed, the error information is as follows: [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for ml.dmlc:xgboost4j:jar:0.5 [WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:exec-maven-plugin is missing. @ line 32, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] xgboost-jvm [INFO] xgboost4j [INFO] xgboost4j-spark [INFO] xgboost4j-flink [INFO] xgboost4j-example [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building xgboost-jvm 0.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- scalastyle-maven-plugin:0.8.0:check (checkstyle) @ xgboost-jvm --- [WARNING] sourceDirectory is not specified or does not exist value=/Users/huzikun/xgboost/jvm-packages/src/main/scala [WARNING] testSourceDirectory is not specified or does not exist value=/Users/huzikun/xgboost/jvm-packages/src/test/scala Processed 0 file(s) Found 0 errors Found 0 warnings Found 0 infos Finished in 67 ms [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ xgboost-jvm --- [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (compile) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.2:testCompile (test-compile) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ xgboost-jvm --- Discovery starting. Discovery completed in 35 milliseconds. Run starting. Expected test count is: 0 DiscoverySuite: Run completed in 97 milliseconds. Total number of tests run: 0 Suites: completed 1, aborted 0 Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 No tests were executed. [INFO] [INFO] --- maven-assembly-plugin:2.6:single (make-assembly) @ xgboost-jvm --- [INFO] Assemblies have been skipped per configuration of the skipAssembly parameter. [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building xgboost4j 0.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- scalastyle-maven-plugin:0.8.0:check (checkstyle) @ xgboost4j --- Processed 7 file(s) Found 0 errors Found 0 warnings Found 0 infos Finished in 488 ms [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ xgboost4j --- [INFO] [INFO] --- exec-maven-plugin:1.5.0:exec (native) @ xgboost4j --- build java wrapper clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -I/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/include -I./java -I/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/include/darwin -shared -o jvm-packages/lib/libxgboost4j.so jvm-packages/xgboost4j/src/native/xgboost4j.cpp build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp make: clang-omp++: No such file or directory make: *\ [jvm-packages/lib/libxgboost4j.so] Error 1 move native lib mv: rename lib/libxgboost4j.so to xgboost4j/src/main/resources/lib/libxgboost4j.dylib: No such file or directory complete [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ xgboost4j --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ xgboost4j --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (compile) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ xgboost4j --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/huzikun/xgboost/jvm-packages/xgboost4j/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ xgboost4j --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- scala-maven-plugin:3.2.2:testCompile (test-compile) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ xgboost4j ---

T E S T S

Running TestSuite [ERROR]: No test suite found. Nothing to run org.testng.TestNGException: Failure in JUnit mode for class ml.dmlc.xgboost4j.java.BoosterImplTest: could not create/run JUnit test suite: cannot retrieve JUnit method at org.testng.junit.JUnitTestRunner.runFailed(JUnitTestRunner.java:230) at org.testng.junit.JUnitTestRunner.start(JUnitTestRunner.java:223) at org.testng.junit.JUnitTestRunner.run(JUnitTestRunner.java:204) at org.testng.TestRunner$2.run(TestRunner.java:639) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRunJUnit(TestRunner.java:664) at org.testng.TestRunner.run(TestRunner.java:576) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:198) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) org.testng.TestNGException: Failure in JUnit mode for class ml.dmlc.xgboost4j.java.DMatrixTest: could not create/run JUnit test suite: cannot retrieve JUnit method at org.testng.junit.JUnitTestRunner.runFailed(JUnitTestRunner.java:230) at org.testng.junit.JUnitTestRunner.start(JUnitTestRunner.java:223) at org.testng.junit.JUnitTestRunner.run(JUnitTestRunner.java:204) at org.testng.TestRunner$2.run(TestRunner.java:639) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRunJUnit(TestRunner.java:664) at org.testng.TestRunner.run(TestRunner.java:576) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:198) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.261 sec - in TestSuite

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ xgboost4j --- Discovery starting. Discovery completed in 96 milliseconds. Run starting. Expected test count is: 7 ScalaBoosterImplSuite: 七月 07, 2016 12:29:26 上午 ml.dmlc.xgboost4j.java.NativeLibLoader smartLoad 严重: failed to load library from both native path and jar 七月 07, 2016 12:29:26 上午 ml.dmlc.xgboost4j.java.DMatrix 严重: load native library failed. 七月 07, 2016 12:29:26 上午 ml.dmlc.xgboost4j.java.DMatrix 严重: java.io.FileNotFoundException: File /lib/libxgboost4j.dylib was not found inside JAR. * RUN ABORTED * java.lang.UnsatisfiedLinkError: ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Ljava/lang/String;I[J)I at ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Native Method) at ml.dmlc.xgboost4j.java.DMatrix.(DMatrix.java:83) at ml.dmlc.xgboost4j.scala.DMatrix.(DMatrix.scala:31) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply$mcV$sp(ScalaBoosterImplSuite.scala:81) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply(ScalaBoosterImplSuite.scala:80) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply(ScalaBoosterImplSuite.scala:80) at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) ... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] xgboost-jvm ........................................ SUCCESS [ 2.417 s] [INFO] xgboost4j .......................................... FAILURE [ 2.786 s] [INFO] xgboost4j-spark .................................... SKIPPED [INFO] xgboost4j-flink .................................... SKIPPED [INFO] xgboost4j-example .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.308 s [INFO] Finished at: 2016-07-07T00:29:26+08:00 [INFO] Final Memory: 23M/448M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project xgboost4j: There are test failures -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :xgboost4j

CodingCat commented 8 years ago

so, when you met error, go back to the output log, you will find the reason

make: clang-omp++: No such file or directory

your system does not have a complete compiler tool chain, would you install that before you install xgboost?

zig-kwin-hu commented 8 years ago

I'm totally new to java and mvn. Could you suggest which complier tool chain should I install? Actually, I don't know what is complier tool chain. Thanks a lot.

CodingCat commented 8 years ago

you can install clang (http://llvm.org/releases/download.html#3.8.0)

zig-kwin-hu commented 8 years ago

I installed clang-mop by running brew install clang-omp. But there is still an error:

1 error generated. make: *\ [jvm-packages/lib/libxgboost4j.so] Error 1 move native lib mv: rename lib/libxgboost4j.so to xgboost4j/src/main/resources/lib/libxgboost4j.dylib: No such file or directory complete

Why does this this error happen? Here is the full report below:

[INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for ml.dmlc:xgboost4j:jar:0.5 [WARNING] 'build.plugins.plugin.version' for org.codehaus.mojo:exec-maven-plugin is missing. @ line 32, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] xgboost-jvm [INFO] xgboost4j [INFO] xgboost4j-spark [INFO] xgboost4j-flink [INFO] xgboost4j-example [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building xgboost-jvm 0.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- scalastyle-maven-plugin:0.8.0:check (checkstyle) @ xgboost-jvm --- [WARNING] sourceDirectory is not specified or does not exist value=/Users/huzikun/xgboost/jvm-packages/src/main/scala [WARNING] testSourceDirectory is not specified or does not exist value=/Users/huzikun/xgboost/jvm-packages/src/test/scala Processed 0 file(s) Found 0 errors Found 0 warnings Found 0 infos Finished in 76 ms [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ xgboost-jvm --- [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (compile) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scala-maven-plugin:3.2.2:testCompile (test-compile) @ xgboost-jvm --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost-jvm:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] No sources to compile [INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ xgboost-jvm --- Discovery starting. Discovery completed in 36 milliseconds. Run starting. Expected test count is: 0 DiscoverySuite: Run completed in 100 milliseconds. Total number of tests run: 0 Suites: completed 1, aborted 0 Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0 No tests were executed. [INFO] [INFO] --- maven-assembly-plugin:2.6:single (make-assembly) @ xgboost-jvm --- [INFO] Assemblies have been skipped per configuration of the skipAssembly parameter. [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building xgboost4j 0.5 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- scalastyle-maven-plugin:0.8.0:check (checkstyle) @ xgboost4j --- Processed 7 file(s) Found 0 errors Found 0 warnings Found 0 infos Finished in 489 ms [INFO] [INFO] --- maven-checkstyle-plugin:2.17:check (checkstyle) @ xgboost4j --- [INFO] [INFO] --- exec-maven-plugin:1.5.0:exec (native) @ xgboost4j --- build java wrapper clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -I/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/include -I./java -I/Library/Java/JavaVirtualMachines/jdk1.8.0_77.jdk/Contents/Home/include/darwin -shared -o jvm-packages/lib/libxgboost4j.so jvm-packages/xgboost4j/src/native/xgboost4j.cpp build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp In file included from jvm-packages/xgboost4j/src/native/xgboost4j.cpp:15: include/xgboost/c_api.h:16:10: fatal error: 'stdio.h' file not found

include 

     ^

1 error generated. make: *\ [jvm-packages/lib/libxgboost4j.so] Error 1 move native lib mv: rename lib/libxgboost4j.so to xgboost4j/src/main/resources/lib/libxgboost4j.dylib: No such file or directory complete [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ xgboost4j --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 1 resource [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (default) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ xgboost4j --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- scala-maven-plugin:3.2.2:compile (compile) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ xgboost4j --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /Users/huzikun/xgboost/jvm-packages/xgboost4j/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ xgboost4j --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- scala-maven-plugin:3.2.2:testCompile (test-compile) @ xgboost4j --- [WARNING] Expected all dependencies to require Scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-compiler:2.10.5 requires scala version: 2.10.5 [WARNING] org.scala-lang:scala-reflect:2.10.5 requires scala version: 2.10.5 [WARNING] ml.dmlc:xgboost4j:0.5 requires scala version: 2.10.5 [WARNING] org.scalatest:scalatest_2.10:2.2.6 requires scala version: 2.10.6 [WARNING] Multiple versions of scala libraries detected! [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ xgboost4j ---

T E S T S

Running TestSuite [ERROR]: No test suite found. Nothing to run org.testng.TestNGException: Failure in JUnit mode for class ml.dmlc.xgboost4j.java.BoosterImplTest: could not create/run JUnit test suite: cannot retrieve JUnit method at org.testng.junit.JUnitTestRunner.runFailed(JUnitTestRunner.java:230) at org.testng.junit.JUnitTestRunner.start(JUnitTestRunner.java:223) at org.testng.junit.JUnitTestRunner.run(JUnitTestRunner.java:204) at org.testng.TestRunner$2.run(TestRunner.java:639) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRunJUnit(TestRunner.java:664) at org.testng.TestRunner.run(TestRunner.java:576) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:198) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) org.testng.TestNGException: Failure in JUnit mode for class ml.dmlc.xgboost4j.java.DMatrixTest: could not create/run JUnit test suite: cannot retrieve JUnit method at org.testng.junit.JUnitTestRunner.runFailed(JUnitTestRunner.java:230) at org.testng.junit.JUnitTestRunner.start(JUnitTestRunner.java:223) at org.testng.junit.JUnitTestRunner.run(JUnitTestRunner.java:204) at org.testng.TestRunner$2.run(TestRunner.java:639) at org.testng.TestRunner.runWorkers(TestRunner.java:1030) at org.testng.TestRunner.privateRunJUnit(TestRunner.java:664) at org.testng.TestRunner.run(TestRunner.java:576) at org.testng.SuiteRunner.runTest(SuiteRunner.java:331) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:326) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:288) at org.testng.SuiteRunner.run(SuiteRunner.java:193) at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:910) at org.testng.TestNG.runSuitesLocally(TestNG.java:879) at org.testng.TestNG.run(TestNG.java:787) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:132) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:198) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:147) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121) Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.262 sec - in TestSuite

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] [INFO] --- scalatest-maven-plugin:1.0:test (test) @ xgboost4j --- Discovery starting. Discovery completed in 107 milliseconds. Run starting. Expected test count is: 7 ScalaBoosterImplSuite: 七月 08, 2016 11:54:35 上午 ml.dmlc.xgboost4j.java.NativeLibLoader smartLoad 严重: failed to load library from both native path and jar 七月 08, 2016 11:54:35 上午 ml.dmlc.xgboost4j.java.DMatrix 严重: load native library failed. 七月 08, 2016 11:54:35 上午 ml.dmlc.xgboost4j.java.DMatrix 严重: java.io.FileNotFoundException: File /lib/libxgboost4j.dylib was not found inside JAR. * RUN ABORTED * java.lang.UnsatisfiedLinkError: ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Ljava/lang/String;I[J)I at ml.dmlc.xgboost4j.java.XGBoostJNI.XGDMatrixCreateFromFile(Native Method) at ml.dmlc.xgboost4j.java.DMatrix.(DMatrix.java:83) at ml.dmlc.xgboost4j.scala.DMatrix.(DMatrix.scala:31) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply$mcV$sp(ScalaBoosterImplSuite.scala:81) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply(ScalaBoosterImplSuite.scala:80) at ml.dmlc.xgboost4j.scala.ScalaBoosterImplSuite$$anonfun$1.apply(ScalaBoosterImplSuite.scala:80) at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at org.scalatest.Transformer.apply(Transformer.scala:22) ... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] xgboost-jvm ........................................ SUCCESS [ 2.490 s] [INFO] xgboost4j .......................................... FAILURE [ 3.427 s] [INFO] xgboost4j-spark .................................... SKIPPED [INFO] xgboost4j-flink .................................... SKIPPED [INFO] xgboost4j-example .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 6.024 s [INFO] Finished at: 2016-07-08T11:54:35+08:00 [INFO] Final Memory: 28M/453M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.scalatest:scalatest-maven-plugin:1.0:test (test) on project xgboost4j: There are test failures -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :xgboost4j huzikundeMacBook-Pro:jvm-packages huzikun$ brew install clang-omp

zig-kwin-hu commented 8 years ago

I guess the problem above is because that I didn't successfully build the shared library. So I delete my xgboost document and restart from the beginning. I download xgboost again. But when I am running make -j4, it shows error. (I have set CC=clang-omp++; CXX=clang-omp++; and I have installed clang-omp by run: brew install clang-mop and I have Xcode on my mac)

clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -DDISABLE_OPENMP -MM -MT build/learner.o src/learner.cc >build/learner.d clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -DDISABLE_OPENMP -MM -MT build/logging.o src/logging.cc >build/logging.d clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -DDISABLE_OPENMP -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -DDISABLE_OPENMP -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d In file included from src/c_api/c_api_error.cc:6: In file included from src/c_api/./c_api_error.h:9: dmlc-core/include/dmlc/base.h:136:10: fatal error: 'sys/types.h' file not foundIn file included from src/c_api/c_api.cc:3: In file included from include/xgboost/data.h:10: dmlc-core/include/dmlc/base.h:136:In file included from 10src/logging.cc:: 7: In file included from fatal errorinclude/xgboost/logging.h: :11: In file included from 'sys/types.h'dmlc-core/include/dmlc/logging.h: 10file: not/usr/local/Cellar/clang-omp/2015-04-01/libexec/bin/../include/c++/v1/cstdio#include <sys/types.h> :found100 ^ : 10: fatal error: In file included from src/learner.cc'stdio.h': 7file: In file included from notinclude/xgboost/logging.h :found11:

In file included from dmlc-core/include/dmlc/logging.h:10: /usr/local/Cellar/clang-omp/2015-04-01/libexec/bin/../include/c++/v1/cstdio:100#include <sys/types.h>: 10: ^ fatal error: 'stdio.h'#include file ^not found

include 

     ^

1 error generated. make: * [build/c_api/c_api_error.o] Error 1 make: * Waiting for unfinished jobs.... 1 error generated. make: * [build/logging.o] Error 1 1 error generated. make: * [build/c_api/c_api.o] Error 1 1 error generated. make: *\ [build/learner.o] Error 1

CodingCat commented 8 years ago

it seems that your system does not even have stdio.h,

I'm not sure why this happens and it has beyond the scope of xgboost....

CodingCat commented 8 years ago

you'd ensure that you can do normal c++ dev work in your system....

zig-kwin-hu commented 8 years ago

When I'm using Xcode, it is easy to include stdio.h.....

zig-kwin-hu commented 8 years ago

and what does it mean by: make: *\ [jvm-packages/lib/libxgboost4j.so] Error 1 move native lib mv: rename lib/libxgboost4j.so to xgboost4j/src/main/resources/lib/libxgboost4j.dylib: No such file or directory complete

CodingCat commented 8 years ago 
When I'm using Xcode, it is easy to include stdio.h.....

yes, so you need to ensure that your local env enables you to do the basic dev work outside of IDE, like vi, 

and what does it mean by:
make: *** [jvm-packages/lib/libxgboost4j.so] Error 1
move native lib
mv: rename lib/libxgboost4j.so to xgboost4j/src/main/resources/lib/libxgboost4j.dylib: No such file or directory
complete

it means that you did not compile native lib correctly

shan-lin commented 8 years ago

I encountered the same error as yours, and solved by the following command as stated in the installation guide: brew install gcc --without-multilib The detailed installation step:

zig-kwin-hu commented 8 years ago

Thank you, I have solved this problem in a similar way as you did.