google-research / dex-lang

Research language for array processing in the Haskell/ML family
BSD 3-Clause "New" or "Revised" License
1.58k stars 107 forks source link

Do unsigned remainder on unsigned types #1211

Open axch opened 1 year ago

axch commented 1 year ago

Manifestation:

big = 2147483647
rem (big + 1) big
> 4294967295

Expected the answer to be 1.

(Note that big + 1 doesn't overflow the underlying Word32 -- it's just the rem operation messing up).

The cause is https://github.com/google-research/dex-lang/blob/c996e988d07f4bec0495136c9d4ca51bcb0c431c/src/lib/ImpToLLVM.hs#L614, which unconditionally emits a signed remainder instruction, regardless of whether the Dex type is meant to be signed or unsigned.

axch commented 1 year ago

By inspection, I expect IDiv to have the same problem.