search

jpmml / jpmml-sklearn

Java library and command-line application for converting Scikit-Learn pipelines to PMML
GNU Affero General Public License v3.0
531 stars 117 forks source link

Test failures during install on MacOS Monterey 12.6.5 #186

Closed kuanghan closed 1 year ago

kuanghan commented 1 year ago

Hi, I cloned this repo and attempted to install, but ran into the following test error:

[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running sklearn2pmml.pipeline.PMMLPipelineTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 s - in sklearn2pmml.pipeline.PMMLPipelineTest
[INFO] Running sklearn2pmml.preprocessing.ExpressionTransformerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.178 s - in sklearn2pmml.preprocessing.ExpressionTransformerTest
[INFO] Running org.jpmml.sklearn.testing.TokenizerTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.179 s - in org.jpmml.sklearn.testing.TokenizerTest
[INFO] Running org.jpmml.sklearn.testing.OutlierDetectorTest
Conflict{id=40, arguments={CRIM=0.03359, ZN=75.00, INDUS=2.950, CHAS=0, NOX=0.4280, RM=7.0240, AGE=15.80, DIS=5.4011, RAD=3, TAX=252.0, PTRATIO=18.30, B=395.62, LSTAT=1.98, MEDV=34.90}, differing (expected vs. actual)={decisionFunction=(0.028890975887470915, 0.02889097588747802)}}
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.332 s <<< FAILURE! - in org.jpmml.sklearn.testing.OutlierDetectorTest
[ERROR] evaluateOneClassSVMHousing(org.jpmml.sklearn.testing.OutlierDetectorTest)  Time elapsed: 0.302 s  <<< FAILURE!
java.lang.AssertionError: Found 1 conflict(s)
        at org.junit.Assert.fail(Assert.java:89)
        at org.jpmml.evaluator.testing.BatchTest.evaluate(BatchTest.java:43)
        at org.jpmml.converter.testing.ModelEncoderBatchTest.evaluate(ModelEncoderBatchTest.java:52)
        at org.jpmml.converter.testing.ModelEncoderBatchTest.evaluate(ModelEncoderBatchTest.java:37)
        at org.jpmml.evaluator.testing.ArchiveBatchTest.evaluate(ArchiveBatchTest.java:63)
        at org.jpmml.evaluator.testing.ArchiveBatchTest.evaluate(ArchiveBatchTest.java:45)
        at org.jpmml.sklearn.testing.OutlierDetectorTest.evaluateOneClassSVMHousing(OutlierDetectorTest.java:82)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
        at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
        at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
        at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
        at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

[INFO] Running org.jpmml.sklearn.testing.RegressorTest
[INFO] Tests run: 47, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.177 s - in org.jpmml.sklearn.testing.RegressorTest
[INFO] Running org.jpmml.sklearn.testing.BSplineTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.033 s - in org.jpmml.sklearn.testing.BSplineTest
[INFO] Running org.jpmml.sklearn.testing.ClustererTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.131 s - in org.jpmml.sklearn.testing.ClustererTest
[INFO] Running org.jpmml.sklearn.testing.ClassifierTest
[INFO] Tests run: 74, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.704 s - in org.jpmml.sklearn.testing.ClassifierTest
[INFO] Running org.jpmml.sklearn.testing.SkLearn2PMMLTest
[INFO] Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.996 s - in org.jpmml.sklearn.testing.SkLearn2PMMLTest
[INFO] Running sklearn.preprocessing.RobustScalerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in sklearn.preprocessing.RobustScalerTest
[INFO] Running sklearn.preprocessing.OneHotEncoderTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 s - in sklearn.preprocessing.OneHotEncoderTest
[INFO] Running sklearn.preprocessing.StandardScalerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in sklearn.preprocessing.StandardScalerTest
[INFO] Running sklearn.preprocessing.LabelBinarizerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in sklearn.preprocessing.LabelBinarizerTest
[INFO] Running sklearn.preprocessing.MinMaxScalerTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in sklearn.preprocessing.MinMaxScalerTest
[INFO] Running sklearn.preprocessing.ImputerTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in sklearn.preprocessing.ImputerTest
[INFO] Running sklearn.SkLearnUtilTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in sklearn.SkLearnUtilTest
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   OutlierDetectorTest.evaluateOneClassSVMHousing:82->ArchiveBatchTest.evaluate:45->ArchiveBatchTest.evaluate:63->ModelEncoderBatchTest.evaluate:37->ModelEncoderBatchTest.evaluate:52->BatchTest.evaluate:43 Found 1 conflict(s)
[INFO] 
[ERROR] Tests run: 161, Failures: 1, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for JPMML-SkLearn 1.7-SNAPSHOT:
[INFO] 
[INFO] JPMML-SkLearn ...................................... SUCCESS [  2.912 s]
[INFO] JPMML SkLearn converter ............................ FAILURE [ 47.945 s]
[INFO] JPMML SkLearn extensions converter ................. SKIPPED
[INFO] JPMML SkLearn H2O.ai converter ..................... SKIPPED
[INFO] JPMML SkLearn LightGBM converter ................... SKIPPED
[INFO] JPMML SkLearn StatsModels converter ................ SKIPPED
[INFO] JPMML SkLearn XGBoost converter .................... SKIPPED
[INFO] JPMML SkLearn converter example applications ....... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  51.260 s
[INFO] Finished at: 2023-04-28T16:14:25-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project pmml-sklearn: There are test failures.
[ERROR] 
[ERROR] Please refer to /Users/kuanghan.huang/Github/jpmml-sklearn/pmml-sklearn/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :pmml-sklearn

Here are some info about my system:

Can anyone help me with this? I'm not that familiar with java at all (I mostly use Python) but I'd like to use this to serialize scikit-learn preprocessing pipelines. Thanks!

vruusmann commented 1 year ago

0.028890975887470915 vs. 0.02889097588747802

The difference is in ~13th -- 14th decimal place, which indicates some "external" numerical stability issue. Could be caused by MacOS, could be caused by OpenJDK or its versions.

The official GitHub Actions CI build runs on GNU/Linux, with JDK versions 8, 11 and 17: https://github.com/jpmml/jpmml-sklearn/blob/master/.github/workflows/maven.yml

Maybe it would be good to add more alternative builds there, such as Windows and MacOS.

But then again, you're not supposed to build the JPMML-SkLearn command-line application from source checkout for everyday use. There's a perfectly good collection of pre-built applications available under the "Releases" section: https://github.com/jpmml/jpmml-sklearn/releases

In the meantime, you can disable Apache Maven unit tests like this:

$ mvn -Dmaven.test.skip=true clean install
kuanghan commented 1 year ago

Thank you so much for the fast reply! I will check out the pre-built JAR files.