Loss of Precision due to long/double conversion

unitsofmeasurement / indriya

JSR 385 - Reference Implementation

Other

122 stars 42 forks source link

Loss of Precision due to long/double conversion #182

Closed kfp closed 5 years ago

kfp commented 5 years ago

The following fails seemingly due to loss of precision over the RationalConverters switching between long/doubles.

val value = new Long(1548351392775L);
val mili = NumberQuantity.of(value, MetricPrefix.MILLI(SECOND));
val nanoLong = mili.to(MetricPrefix.NANO(SECOND)).getValue().longValue();
assertEquals(value * 1000000, nanoLong);

Is there a different pattern which should be followed for big longs?

keilw commented 5 years ago

As seen in this demo for Java 10 (the example seems Kotlin) it works if you explicitly use BigInteger.