search

thewisenerd / python-on-a-chip

Automatically exported from code.google.com/p/python-on-a-chip
Other
0 stars 0 forks source link

Fix float comparisons #81

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
ipm> a=3.4e-15
 ipm> a
 0.000000
ipm> a == 0.0
 True
ipm> a * 1.0e15
 3.400000

What is the expected output? What do you see instead?
'a' prints as zero, but it is not zero and should not compare positively to 
it.

Original issue reported on code.google.com by dwhall...@gmail.com on 13 Mar 2010 at 9:11

GoogleCodeExporter commented 9 years ago 
Created system test t281 which serves as a regression for this defect:

./t281.out
a == b ?  True 
Error:     0xE4
  Release: 0x08
  FileId:  0x09
  LineNum: 1656
Traceback (top first):
  t281()
  <module>.

Original comment by dwhall...@gmail.com on 14 Mar 2010 at 5:29

GoogleCodeExporter commented 9 years ago 
Edited float.c to fix the COMP_EQ and COMP_NE cases.  Had to change a test in 
t158 due 
to a float-rounding issue.  Also added a test where the rounding works out as 
expected

Original comment by dwhall...@gmail.com on 14 Mar 2010 at 3:11

GoogleCodeExporter commented 9 years ago 
System tests pass.  Mainlined directly, see r459

Original comment by dwhall...@gmail.com on 14 Mar 2010 at 3:12