search

EightAndAHalfTails / eee3-imgtec-fpu

0 stars 0 forks source link

[dot2] inf, zero bug #162

Closed reginalio closed 10 years ago

reginalio commented 10 years ago 
Warning: 2D dot product of 1:00000000:00010111100110111111110, 0:11111111:00000000000000000000000, 1:10110011:10100111000110100110010 and 1:11111100:11100110100010110001000gives 0:11111111:00000000000000000000000 which is incorrect. Correct answer is NAN

^ correct answer is NaN or -inf

Warning: 2D dot product of 0:00100110:11001010111111011000001, 1:01010010:10100101001011100110001, 1:11011000:01110010001101010001100 and 0:00000000:00000000000000000000000 gives 1:00101011:00000000000000000000000 which is incorrect. Correct answer is 1:00000000:00000010111100110010011

^ incorrect zero handling

gzf1992 commented 10 years ago

should be fixed by commit 2ef3ea74d57b672f27c1a9c34d49d4e81b938197.

gzf1992 commented 10 years ago

closed #162