search

jpmml / jpmml-evaluator

Java Evaluator API for PMML
GNU Affero General Public License v3.0
892 stars 255 forks source link

Exception in thread "main" java.lang.ArithmeticException: integer overflow ? #175

Closed OnlyFor closed 5 years ago

OnlyFor commented 5 years ago

when i use 1.4.13 example,

java -cp pmml-evaluator-example-executable-1.4.13.jar org.jpmml.evaluator.EvaluationExample --model Model_test8-train-test2_score.xml --input input.csv --output output.csv --missing-values "" --separator ","

Exception in thread "main" java.lang.ArithmeticException: integer overflow
        at java.base/java.lang.Math.toIntExact(Math.java:1071)
        at org.jpmml.evaluator.Functions$22.evaluate(Functions.java:309)
        at org.jpmml.evaluator.functions.RoundingFunction.evaluate(RoundingFunction.java:38)
        at org.jpmml.evaluator.functions.UnaryFunction.evaluate(UnaryFunction.java:43)
        at org.jpmml.evaluator.ExpressionUtil.evaluateFunction(ExpressionUtil.java:474)
        at org.jpmml.evaluator.ExpressionUtil.evaluateApply(ExpressionUtil.java:439)
        at org.jpmml.evaluator.ExpressionUtil.evaluateExpression(ExpressionUtil.java:190)
        at org.jpmml.evaluator.ExpressionUtil.evaluate(ExpressionUtil.java:152)
        at org.jpmml.evaluator.ExpressionUtil.evaluateExpressionContainer(ExpressionUtil.java:84)
        at org.jpmml.evaluator.ExpressionUtil.evaluateTypedExpressionContainer(ExpressionUtil.java:89)
        at org.jpmml.evaluator.ExpressionUtil.evaluate(ExpressionUtil.java:112)
        at org.jpmml.evaluator.ModelEvaluationContext.resolve(ModelEvaluationContext.java:98)
        at org.jpmml.evaluator.EvaluationContext.evaluate(EvaluationContext.java:114)
        at org.jpmml.evaluator.ExpressionUtil.evaluateFieldRef(ExpressionUtil.java:214)
        at org.jpmml.evaluator.ExpressionUtil.evaluateExpression(ExpressionUtil.java:166)
        at org.jpmml.evaluator.ExpressionUtil.evaluate(ExpressionUtil.java:152)
        at org.jpmml.evaluator.ExpressionUtil.evaluateApply(ExpressionUtil.java:418)
        at org.jpmml.evaluator.ExpressionUtil.evaluateExpression(ExpressionUtil.java:190)
        at org.jpmml.evaluator.ExpressionUtil.evaluate(ExpressionUtil.java:152)
        at org.jpmml.evaluator.ExpressionUtil.evaluateExpressionContainer(ExpressionUtil.java:84)
        at org.jpmml.evaluator.ExpressionUtil.evaluateTypedExpressionContainer(ExpressionUtil.java:89)
        at org.jpmml.evaluator.ExpressionUtil.evaluate(ExpressionUtil.java:112)
        at org.jpmml.evaluator.ModelEvaluationContext.resolve(ModelEvaluationContext.java:98)
        at org.jpmml.evaluator.EvaluationContext.evaluate(EvaluationContext.java:114)
        at org.jpmml.evaluator.PredicateUtil.evaluateSimplePredicate(PredicateUtil.java:137)
        at org.jpmml.evaluator.PredicateUtil.evaluatePredicate(PredicateUtil.java:83)
        at org.jpmml.evaluator.PredicateUtil.evaluate(PredicateUtil.java:73)
        at org.jpmml.evaluator.PredicateUtil.evaluatePredicateContainer(PredicateUtil.java:63)
        at org.jpmml.evaluator.scorecard.ScorecardEvaluator.evaluateRegression(ScorecardEvaluator.java:114)
        at org.jpmml.evaluator.ModelEvaluator.evaluateInternal(ModelEvaluator.java:678)
        at org.jpmml.evaluator.ModelEvaluator.evaluate(ModelEvaluator.java:580)
        at org.jpmml.evaluator.EvaluationExample.execute(EvaluationExample.java:373)
        at org.jpmml.evaluator.Example.execute(Example.java:90)
        at org.jpmml.evaluator.EvaluationExample.main(EvaluationExample.java:225)

but everything just fine in 1.4.7 ? files.zip

i use round in scorecard model

vruusmann commented 5 years ago

Exception in thread "main" java.lang.ArithmeticException: integer overflow

The JPMML-Evaluator library uses Java's 32-bit int/java.lang.Integer to represent PMML integer data type values. This is fine for the majority of cases, but there seem to be a number edge cases where Java's 64-bit long/java.lang.Long would be required instead.

In your case, one of the inputs is a PMML double value, does not fit into Java's java.lang.Integer after rounding.

but everything just fine in 1.4.7?

The most likely explanation is that the 1.4.7 version simply didn't perform integer bounds checking (which lead to incorrect prediction).

Closing this issue as a duplicate of https://github.com/jpmml/jpmml-evaluator/issues/141

vruusmann commented 5 years ago

@OnlyFor In your code there's an expression x_int = round(x_double) / 1000; the integer overflow exception happens during the round(x_double) part.

Wouldn't it be possible (ie. functionally identical given the true nature of the x_double variable) to do the division by 1000 before rounding (x_int = round(x_double / 1000d))?

OnlyFor commented 5 years ago

But I am pretty sure the results of 1.4.7 is correct, and the reason of * & / 1000 is to keep 3 decimal after rounding, as same as round(x, 3) in other languages

vruusmann commented 5 years ago

But I am pretty sure the results of 1.4.7 is correct

Perhaps the integer overflow happened with reason codes that did not emerge as the "winner" in the end.

You could tweak the current 1.4-SNAPSHOT codebase, and simply do return result (instead of return Math.toIntExact(result)) here: https://github.com/jpmml/jpmml-evaluator/blob/master/pmml-evaluator/src/main/java/org/jpmml/evaluator/Functions.java#L309

Are you still seeing successful/correct predictions then?

the reason of * & / 1000 is to keep 3 decimal after rounding, as same as round(x, 3) in other languages

The PMML standard only provides "fixed precision" round function. To emulate arbitrary precision rounding, then you could try using the formatNumber function instead: http://dmg.org/pmml/v4-3/BuiltinFunctions.html#formatnumber

Something like this:

<DerivedField name="round(x, 3)" dataType="double">
  <Apply function="formatNumber">
    <FieldRef field="x"/>
    <Constant dataType="string">0.000</Constant> <!-- check this pattern! -->
  </Apply>
</DerivedField>
OnlyFor commented 5 years ago

thx !

i fix this problem by using something like:

%.3f