error in Dubugdivision

[cyckun]

Hi, i find the division test result is error, everybody meet such problem? In the test, a = 3.2 b=9.7, the result should be a / b = 0.32, but the code result is 0.0198, Is there some errors in my test?

[snwagh]

I do believe that could be a result of the finite amount of precision in storing numbers. I think if the integer part is generally of this order (about 4 or beyond), the correctness would be affected.

You can try

• Increasing the bit-width by changing to uint64_t
• Reducing the fixed-point precision

[cyckun]

great，using uint64_t solved my question, thanks you, snwagh.