search

ucb-bar / berkeley-hardfloat

Other
291 stars 87 forks source link

Fix resizeRawFloat preserving the correct sExp for NaN/Infty inputs #85

Closed jerryz123 closed 2 months ago

jerryz123 commented 2 months ago

The sAdjustedSexp calculation does not correctly represent the sExp field when the input is NaN or infinity. This corrects that, I believe.

jerryz123 commented 2 months ago

I'm using resizeRawFloat to upconvert sub-F64 values such that I can use the same F64 RecFNToIN to do IEEE fp-to-int conversions.

The problem I'm seeing is that the conversion of a rawFN to a recFN expressed here: https://github.com/ucb-bar/berkeley-hardfloat/blob/26f00d00c3f3f57480065e02bfcfde3d3b41ec51/hardfloat/src/main/scala/recFNFromFN.scala#L47-L51 does not construct a correct recFN representation of a infinity or NaN when the input rawFloat is resized up (the high non-sign bits of the exponent are not 11).

Perhaps this expression is not intended to work with resized rawFNs?

aswaterman commented 2 months ago

Yeah, that expression is only valid in its context; it is (legitimately) making assumptions about what rawFloatFromFN might return.