search

calimero-project / calimero-core

Core library for KNX network access and management
Other
129 stars 65 forks source link

DPTXlator2ByteFloat: inaccurate precision #7

Closed Snickermicker closed 10 years ago

Snickermicker commented 10 years ago

DPTXlator2ByteFloat cannot represent the maximum and minimum values for this type. Root cause seems to be in private method: fromDPT(). The accuracy when calculating the value is Float but might need to be Double. The following test cases test the erroneous calculation: 0x7FFF should result in 670760.96 but results in 670760.94 and 0xF800 should result in -671088.64 but results in -671088.6.

aschamberger commented 10 years ago

Shouln't DPTXlator4ByteFloat be also changed to double now to be consistent?

bmalinowsky commented 10 years ago

The KNX 4-byte float is actually more reasonably specified using IEEE 754. With Java implementing that standard directly in float (or better speaking, a subset thereof), the type match is as good as it gets. The problem with the 2-byte float was, that--although way less precise--its encoding allows some values that cannot be represented using Java single-precision format. As pointed out by the OP, that can be a problem, e.g., when checking the special value for "invalid data".