Open aibohang opened 2 years ago
Apologies for the delay, have been on time off. This is somewhat expected as you might be reaching the precision limit. With 64-bit data type, you have about 32 bits of "space" before things overflow in a multiplication. If you use say a precision of 26 bits, that leaves about 6 bits of room for the integer part of any value. Thus, as you're approaching numbers whose integer part is about 2^6 = 64, you might have problems. 7.7338 * 6.7338 = 52.07786244 is in the ball park.
Try reducing the fixed-point precision of your computation. For example, you might be able to do 7.7 * 6.7 in about 11 bits of fixed-point precision or so (off the top of my head, you'll have to test this out).
If two floating point numbers float f = 7.7338 and float e = 6.7338,First, scale the two numbers, then call matrix multiplication funcMatMul (), and find that the product is wrong.
However, if float f = 2.7338 and float e = 3.7338, the result of the product is correct. Does funcmatmul() require the size range of numbers?