CME PRICE9 double precision issue

[swarwick]

I we get a PRICE9 value from CME and use the SbeDouble to try and convert the value to a double you can get precision/scaling errors. For example the value for price as a long from the feed is 259575000000000l and the exponent is -9 then the code currently does: public double asDouble() { return getMantissa() Math.pow(10, getExponent()); } which with values filled in is: public double asDouble() { return 259575000000000l Math.pow(10, -9); } the issue is the double value returned as a precision issue and is actually: 259575.00000000003

The only way I can see to get around the precision problem so far is to use a BigDecimal to set the scale and return that double like: return new BigDecimal(259575000000000l * Math.pow(10, -9)).setScale(-9, , RoundingMode.HALF_UP).doubleValue();

Is there another way to resolve this or is it just a known issue that will not be resolved?

[iamolever]

That’s why we have two components of double values. It’s impossible to use primitive double in all cases. You have to use objects in business logic if errors can’t be ignored.

Отправлено с iPad

11 янв. 2019 г., в 20:45, Steven Warwick notifications@github.com написал(а):

I we get a PRICE9 value from CME and use the SbeDouble to try and convert the value to a double you can get precision/scaling errors. For example the value for price as a long from the feed is 259575000000000l and the exponent is -9 then the code currently does: public double asDouble() { return getMantissa() Math.pow(10, getExponent()); } which with values filled in is: public double asDouble() { return 259575000000000l Math.pow(10, -9); } the issue is the double value returned as a precision issue and is actually: 259575.00000000003

The only way I can see to get around the precision problem so far is to use a BigDecimal to set the scale and return that double like: return new BigDecimal(259575000000000l * Math.pow(10, -9)).setScale(-9, , RoundingMode.HALF_UP).doubleValue();

Is there another way to resolve this or is it just a known issue that will not be resolved?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[swarwick]

Fair enough