search

EightAndAHalfTails / eee3-imgtec-fpu

0 stars 0 forks source link

[dot3] overflow bug #161

Closed reginalio closed 10 years ago

reginalio commented 10 years ago 
 Warning: 3D dot product of 1:11111111:00000000000000000000000, 1:01000001:00101011111010011001011, 0:11110000:00011011111010010010100, 1:11111100:01100110110101001010011, 0:00000000:01000011111001110000011 and 0:10111001:00101000010101000100100 gives 1:11111111:00000000000000000000000 which is incorrect. Correct answer is NAN

^Correct answer is either NaN or +inf, please check

Warning: 3D dot product of 1:00000000:00010111100110111111110, 0:11111111:00000000000000000000000, 1:10110011:10100111000110100110010, 1:11111100:11100110100010110001000, 1:00000000:00000000000000000000000 and 1:00000000:11001001100011010111101 gives 0:11111111:00000000000000000000000 which is incorrect. Correct answer is NAN

^Correct answer is either NaN or -inf, please check

Warning: 3D dot product of 0:00000000:00110000001000000101100, 1:10101011:10101101110010110001111, 1:00000000:00000000000000000000000, 1:11100010:11001010010011011000110, 0:00000000:00000000000000000000000 and 1:11101001:01110000000010100010010 gives 1:00110101:00000000000000000000000 which is incorrect. Correct answer is 1:00101011:01000011001100011000110
Warning: 3D dot product of 0:00100110:11001010111111011000001, 1:01010010:10100101001011100110001, 1:11011000:01110010001101010001100, 0:00000000:00000000000000000000000, 0:10110000:10110011110101100010110 and 1:00000000:00010100101001111101011 gives 1:00101111:00101001010100111100010 which is incorrect. Correct answer is 1:00101111:00011001010100111100010
Warning: 3D dot product of 1:00000000:01000000111000011110110, 1:00000000:01100000100010000011010, 0:11000101:10100010011011000100010, 0:00000000:00000000000000000000000, 1:01100000:10101011101100011101010 and 0:00000000:10100110111110011011000 gives 0:00011000:00000000000000000000000 which is incorrect. Correct answer is 1:00000000:00000000000000000000000

^ is there something wrong with zero handling in dot 2? (Issue #162)

test ran was Random50000 (total errors 300+)

gzf1992 commented 10 years ago

Also fixed by the commit 2ef3ea74d57b672f27c1a9c34d49d4e81b938197, which fixes zero handling and infinity handling bug in dot2.

gzf1992 commented 10 years ago

closed #161